-- Logs begin at Tue 2022-08-09 17:06:45 UTC, end at Tue 2022-08-09 18:49:12 UTC. -- Aug 09 17:55:46.893879 dv-cinder-797970-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 17:55:46.905644 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 17:55:46 2022] *** Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: nodename: dv-cinder-797970-5-iscsi Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: machine: x86_64 Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: clock source: unix Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: pcre jit disabled Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: detected number of CPU cores: 4 Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: current working directory: / Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: detected binary path: /usr/bin/uwsgi-core Aug 09 17:55:46.907688 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: your processes number limit is 47680 Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: your memory page size is 4096 bytes Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: detected max file descriptor number: 2048 Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: lock engine: pthread robust mutexes Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: thunder lock: enabled Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 17:55:46.908299 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 17:55:46.949550 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: Python main interpreter initialized at 0x559b2fa329c0 Aug 09 17:55:46.949550 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: python threads support enabled Aug 09 17:55:46.949550 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: your server socket listen backlog is limited to 100 connections Aug 09 17:55:46.949550 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: your mercy for graceful operations on workers is 90 seconds Aug 09 17:55:46.949890 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 17:55:46.949967 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: *** Operational MODE: preforking *** Aug 09 17:55:46.950232 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: *** uWSGI is running in multiple interpreter mode *** Aug 09 17:55:46.950232 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: spawned uWSGI master process (pid: 80529) Aug 09 17:55:46.950435 dv-cinder-797970-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 17:55:46.951831 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: spawned uWSGI worker 1 (pid: 80544, cores: 1) Aug 09 17:55:46.951831 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: spawned uWSGI worker 2 (pid: 80545, cores: 1) Aug 09 17:55:46.951831 dv-cinder-797970-5-iscsi devstack@keystone.service[80529]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 17:55:52.539280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:55:52.539280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: from cryptography.utils import int_from_bytes Aug 09 17:55:52.540006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:55:52.540006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: from cryptography.utils import int_from_bytes Aug 09 17:55:52.541107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:55:52.541107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: from cryptography.utils import int_from_bytes Aug 09 17:55:52.541275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:55:52.541275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: from cryptography.utils import int_from_bytes Aug 09 17:55:52.590188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 17:55:52.590387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 17:55:52.590546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 17:55:52.590639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] command line args: [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 17:55:52.590750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 17:55:52.590896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 17:55:52.590982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] command line args: [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 17:55:52.591096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ================================================================================ {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 17:55:52.591183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 17:55:52.591362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ================================================================================ {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 17:55:52.591447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] admin_token = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.591628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] admin_token = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.591714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] config_dir = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.591873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] config_dir = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.591957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] config_source = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] config_source = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] debug = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] debug = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.592962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.593982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_config_append = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_config_append = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_dir = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_file = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_dir = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.594935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_file = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.595782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.596980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.597944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] publish_errors = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.598870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] publish_errors = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.599956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] standard_threads = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] use_journal = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.600918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] use_json = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] use_stderr = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] use_syslog = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] standard_threads = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.601776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.601853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.602012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.602092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.602272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.602350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.602500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] use_journal = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.602578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.602655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] use_json = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.602760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.602889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] use_stderr = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.603640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.603755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] use_syslog = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.603946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:55:52.604236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.604972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.605929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.606948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.607966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.608906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.609958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.610950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.611805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.612962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.613991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.614920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.615865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.616904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.617914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.618961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.619928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.620957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.621897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.622867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.623968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.624990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.625914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.626892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.external = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.627852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.password = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] auth.token = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.628952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.629981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.630901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.631863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.external = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.632975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.password = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] auth.token = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.633975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.634136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.634791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.635924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.636970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.637920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.638898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.639864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.640949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.641980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.642969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.643890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.644998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.645921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.646983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.647933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.648857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.649014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.649100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.649254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.649334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.649743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.650437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.650594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.650670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.650876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.651933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.652856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.653953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.654922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.655936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.656857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.657955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.658973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.659931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.660873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.661957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] role.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] role.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.662919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.663938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.664952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.665959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.666972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] role.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.667822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] role.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.668995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.669950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.670992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.671881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.672937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.673886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.674867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.connection = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.675959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.676982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.677931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.678900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80545) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 17:55:52.679774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.679974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.680261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.680516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.connection = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.680757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.681000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.681430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.681711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.681945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.682180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.682413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.682650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.682917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.683150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.683392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.683623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.683865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.684098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.684337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.684562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.684796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.685025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:55:52.685192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 17:55:52.709438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.709670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.709900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.710107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.710339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.710563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.720503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.720998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.749133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.749366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.752466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.752698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.803346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.803789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.854720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.854959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.855751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.855988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.920989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.921096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.921179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.921307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.921390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.921521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.944709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.944941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.945962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.946045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.946173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.946387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.957116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.957458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.969978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.970983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.971077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.971199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.971280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80544) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.971433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.971646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80545) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:55:52.988750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.988883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.988965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:55:52.989040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:55:52.989287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.989371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.989486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:55:52.989568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:55:52.989760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.989842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:52.990689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.002549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.002827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.002915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.003101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.003184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.003367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.003449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.003647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.010370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.010993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.013506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.013796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.014119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.014921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.015815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.016827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.040995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.041792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.042498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.042825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.043335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.043630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.047447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.047738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.048030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.048192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.048477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.048759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.054581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.054898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.055272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.055559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.059046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.059343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.059630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.059901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.059985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.060817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.061006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.061097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.061280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.061552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.061820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.076764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.077045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.077624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.077904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.081097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.081860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.084275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.084577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.084995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.085295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.089757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.090505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.092979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.093675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.096147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.096452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.096740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.096836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.097926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.098214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.114271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.114578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.114834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.114922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.115131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.115228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.115402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.115667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.122914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.123424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.124501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.124785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.125066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.125350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.127695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.127986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.128147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.128281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.128432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.128577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.128719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.128859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.129007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.129284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.141215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.141503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.141643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.141786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.141929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.142966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.143069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.143346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.160740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.160842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.163741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.163832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.167412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.167597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.168980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.169152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.170482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.170629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.170815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.170944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.171117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.171224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.177568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.177665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.179193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.179320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.182372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.182468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.182667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.182846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.183003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.183087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.183285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.183377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.191296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.191497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.191593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.191791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.191915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.192074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.199105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80545) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.199203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80544) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:55:53.199393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.199489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.199675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.199757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.199954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.200854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.201971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80544) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.202888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80545) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:55:53.237414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559b2fa329c0 pid: 80545 (default app) Aug 09 17:55:53.240747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154884b4-bf90-4db5-8427-e141dd5bd322 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:53.241036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154884b4-bf90-4db5-8427-e141dd5bd322 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:53.241309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154884b4-bf90-4db5-8427-e141dd5bd322 None None] PATH_INFO: `/v3` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:53.244231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559b2fa329c0 pid: 80544 (default app) Aug 09 17:55:53.246068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Tue Aug 9 17:55:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:01.945683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.946250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.946250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.951168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:01.956387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.956811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.956942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.028195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: INFO dbcounter [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] Registered counter for database keystone Aug 09 17:56:02.049158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG oslo_db.sqlalchemy.engines [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80545) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 17:56:02.055345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writer thread running {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 17:56:02.232299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG passlib.handlers.bcrypt [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80545) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 17:56:02.232717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG passlib.handlers.bcrypt [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80545) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 17:56:02.287109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:02.288697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.418876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 17:56:02.580328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.650846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: INFO dbcounter [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] Registered counter for database keystone Aug 09 17:56:02.670477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG oslo_db.sqlalchemy.engines [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 17:56:02.676691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writer thread running {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 17:56:02.794253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018005d-ab24-4eb2-ab28-f1e1698a7a06 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.798472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.799107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.799384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.799648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.807566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.859714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 17:56:02.859988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95770-0d3b-443d-a679-57aba4e60af3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.869629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:56:02.880454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.889917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25878be-8b10-4874-af9d-926f32aa5ccd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.894366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.895501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.895787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.896058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] PATH_INFO: `/v3/projects/c024e80f00ae4274ad841a20660b3ad2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.897489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] RBAC: Authorizing `identity:get_project(project_id=c024e80f00ae4274ad841a20660b3ad2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.949593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 17:56:02.949935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b718ea2-aec8-4923-98da-15e39938e673 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.954521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/projects/c024e80f00ae4274ad841a20660b3ad2 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:04.949888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7265df-6ec4-4c66-82c4-02e8f720a3b9 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:04.950453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7265df-6ec4-4c66-82c4-02e8f720a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:04.950453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7265df-6ec4-4c66-82c4-02e8f720a3b9 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:04.954491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:04.959869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:04.960251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:04.960415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:04.982796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:04.984286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:04.991970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 17:56:05.014169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7265df-6ec4-4c66-82c4-02e8f720a3b9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:05.027014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7265df-6ec4-4c66-82c4-02e8f720a3b9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:05.031118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:05.031703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.031980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.032249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.039663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:05.042439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac39678-ff19-4a3b-80ac-ee6180806df8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:05.050920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Tue Aug 9 17:56:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:56:05.056215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:05.064916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42c4cb-504e-4a11-b6fa-0e21a532b8f0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:05.069208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:05.069811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.070087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.070356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.071719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:05.073490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c8d79-848d-4f93-9b2d-7c284c781174 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:05.075005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Tue Aug 9 17:56:05 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:07.142006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748540c-8cbe-4e84-a455-4248797f7976 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.142550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748540c-8cbe-4e84-a455-4248797f7976 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.142550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748540c-8cbe-4e84-a455-4248797f7976 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.144053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:07.149219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.149606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.150555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.167125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1e9f-6857-4238-9e33-32b8a5f8203c None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.167409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1e9f-6857-4238-9e33-32b8a5f8203c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.167798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1e9f-6857-4238-9e33-32b8a5f8203c None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.168453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:07.169153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:07.169624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:07.174053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.174330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.174599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.217664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG passlib.handlers.bcrypt [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80544) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 17:56:07.218076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG passlib.handlers.bcrypt [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80544) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 17:56:07.251723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:07.264476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:07.266046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:07.277183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.300768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.365357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:07.383117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b356ee-e707-495e-b6d7-c78b1930c9ec None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.387825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.417629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.418187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.418468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.418782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.430067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:07.439289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:07.440296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:07.441100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:07.446171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.494030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2849a-fcd5-4794-b064-ad4ecedf92e9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.496540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.530489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.533283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.533283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.533283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.533283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.547977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1d90-d623-4bab-9906-16099eb9832a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.583931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.584531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.584807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.585077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.585743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:07.586039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.587635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.599046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:07.599234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.600758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.605870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.700114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:56:07.706187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.709405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.814268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5411-f459-4eb7-b763-a71436528b11 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.817477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec54705-9813-4b28-8a1f-f96c3f55d99c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.847547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.848392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.848673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.848949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.851462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.852073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.852348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.853157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.854393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.855866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.857418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:07.862860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.867538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.869680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.878155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:56:07.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:07.890292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:56:07.891122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:56:07.891122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:56:07.891122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:07.891122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:07.894498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.949722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.962644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a551-56b2-422b-9ac7-7c8d8f266f30 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.992377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d43a5-b766-4b43-b5d2-5c6e7a3573d6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.996517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.997149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.997427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.997697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.998889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.000347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.002084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:08.007776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.028167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.028782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.029063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.029338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.029985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:08.030287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.031711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de4c01-47a2-4375-9bd0-bba0be20e5a2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.049127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:08.090245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:08.112156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a17428-dde6-4eba-befc-655c94f097c8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.148386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.149251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.150301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.150383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.151130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.domain.id=default, user.id=c800e45fdce14a59ae5bf0f84f3671cd) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:08.151429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.152884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71746e5-4b89-4cb5-9590-0d55d8c54f88 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.160177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Tue Aug 9 17:56:08 2022] GET /identity/v3/role_assignments?user.id=c800e45fdce14a59ae5bf0f84f3671cd&scope.domain.id=default&role.id=7409e93d755b445b891c0195b28879f6 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:10.522219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b648dd-2064-4db2-bf25-08f2d47dc507 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.523443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b648dd-2064-4db2-bf25-08f2d47dc507 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.524497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b648dd-2064-4db2-bf25-08f2d47dc507 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.526164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:10.534791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351ed0-7146-4bf5-9ee1-b2191e868fd2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.536602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.537035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.537448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.540856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351ed0-7146-4bf5-9ee1-b2191e868fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.541689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351ed0-7146-4bf5-9ee1-b2191e868fd2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.544819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:10.563570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:10.565152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:10.577495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1870-3b85-443a-b008-0248f4fa6076 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.578018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1870-3b85-443a-b008-0248f4fa6076 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.578437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1870-3b85-443a-b008-0248f4fa6076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.612544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-abbb1870-3b85-443a-b008-0248f4fa6076 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:10.614031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-abbb1870-3b85-443a-b008-0248f4fa6076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:10.703866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 13/24] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:10.705629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b340-8d0b-4a05-94fb-63fcd83d68e4 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.706094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b340-8d0b-4a05-94fb-63fcd83d68e4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.706546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b340-8d0b-4a05-94fb-63fcd83d68e4 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.708769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 14/25] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:10.716524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.717067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.717552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.723087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 12/26] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:10.736116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:10.737560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:10.768979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:10.812089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 15/27] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:10.813577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:10.822609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:10.883080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:10.930347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea174e3-d740-43e1-ab77-155fc6ea05e4 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:10.996068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36263e59-36cb-49ea-a602-2b9f26239978 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:10.999059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:10.999927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.000402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.000880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.022788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.025989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:56:11.045780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:11.046673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:56:11.047495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:56:11.047495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:56:11.047495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:11.049310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 16/28] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:11.050453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.068982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.069633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.069916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.070189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.091579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:11.104486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:11.105404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:11.106739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 13/29] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:11.107405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.128479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:11.167788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851b74f-8ffd-485c-bf49-e022b8ef0db9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.203658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.204662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.205140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.205624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.206483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.208125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8631d6-933e-47e2-a2a3-56079c0f3da4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.438541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d97bda-174c-45be-a0f6-eadfdb45fe97 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.451147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:11.477852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.477852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.477852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.477852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.478190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:11.478190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.478594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.490533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 14/30] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:11.491631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.497516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 17/31] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Aug 9 17:56:11 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:11.499328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163a35-ef24-4384-9e83-141be65b61bc None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.499824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163a35-ef24-4384-9e83-141be65b61bc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.500327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163a35-ef24-4384-9e83-141be65b61bc None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.501780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 18/32] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:11.502768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f163a35-ef24-4384-9e83-141be65b61bc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.609265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94c252-0bbf-4b06-a9e1-9634084dd43e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.634606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f163a35-ef24-4384-9e83-141be65b61bc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.650828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.651568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.651845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.652115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.652796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:11.653095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.654620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c28aae-0a40-4617-946c-37a7361c7312 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.658536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:11.668765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 15/33] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:11.670106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e9fd-fd22-4355-bd9b-1d44710646f5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.670389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e9fd-fd22-4355-bd9b-1d44710646f5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.670658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e9fd-fd22-4355-bd9b-1d44710646f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.681993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.683697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.684429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.685149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.686974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.689208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc09ac-c128-4080-9a70-870117f01f41 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.691042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7c19e9fd-fd22-4355-bd9b-1d44710646f5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:11.691954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7c19e9fd-fd22-4355-bd9b-1d44710646f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:11.719465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:11.767011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 16/34] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:11.768264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eaf892-0af9-49b4-99e4-3b2d5a4c3a34 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.768552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eaf892-0af9-49b4-99e4-3b2d5a4c3a34 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.768815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eaf892-0af9-49b4-99e4-3b2d5a4c3a34 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.770148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 17/35] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:11.770779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eaf892-0af9-49b4-99e4-3b2d5a4c3a34 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.827357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 19/36] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:56:11 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:11.829247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.829532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.829809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.856717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:11.858327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:11.904935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eaf892-0af9-49b4-99e4-3b2d5a4c3a34 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.951833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.952660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.953010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.953220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.954579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 20/37] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:11.955103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.955368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.956701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.958860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 18/38] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:11 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:11.964180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.995471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:12.070466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:12.075576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96157b-d544-4c6c-b0e8-42ad2a31763a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.077326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94ce-2ebd-419c-a18c-670631607f2f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.112307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.112921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.113202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.113505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.114316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.115420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.115755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93040ee-8263-4725-be54-258fe4c66cd7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.116153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.116434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.116707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.117839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.120119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.121694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 19/39] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:12.122431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:12.183911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 21/40] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Aug 9 17:56:11 2022] POST /identity/v3/roles => generated 225 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:12.185365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa0e60-76e7-4168-9d97-b2fc8b740cef None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.185650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa0e60-76e7-4168-9d97-b2fc8b740cef None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.185918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa0e60-76e7-4168-9d97-b2fc8b740cef None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.191466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 22/41] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:12.191466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa0e60-76e7-4168-9d97-b2fc8b740cef None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:12.223460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:12.232119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac54df-c0f6-4088-b24a-00174f774978 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.277924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.278741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.279027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.280183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.280888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.282629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.312789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa0e60-76e7-4168-9d97-b2fc8b740cef None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.350661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.351787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.351787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.352174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] PATH_INFO: `/v3/domains/default/users/c800e45fdce14a59ae5bf0f84f3671cd/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.354909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c800e45fdce14a59ae5bf0f84f3671cd, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.356494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e5c7e-9909-4d96-b7f1-ad533040a6ee None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.435757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 23/42] 10.222.0.36 () {58 vars in 1361 bytes} [Tue Aug 9 17:56:12 2022] PUT /identity/v3/domains/default/users/c800e45fdce14a59ae5bf0f84f3671cd/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:12.437134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.437518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.437892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.449255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 20/43] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Aug 9 17:56:12 2022] POST /identity/v3/roles => generated 223 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:12.458006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:12.459481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:12.476623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:12.544215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 24/44] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:12.567405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:12.685117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6a49-3e34-4248-add2-cba5bb187797 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.719460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.720090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.720375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.720651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.721841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.723439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef05e34-1678-4179-b220-f7341811a1ca None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.725019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 21/45] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:12.731460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:12.827843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e9b5c-7b1a-482c-aa81-55f5264f48c5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.861142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.861745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.862025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.862299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.862978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.864441 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb72e8-e092-4491-a1db-b7a82bdf82cf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.944869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:12.951042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 25/46] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:56:12 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:14.519452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da419e2-4d67-4b0f-87a2-608cc340c4d6 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.520007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da419e2-4d67-4b0f-87a2-608cc340c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.520007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da419e2-4d67-4b0f-87a2-608cc340c4d6 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.521339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 22/47] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:14.527782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.528195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.528361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.546917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:14.548094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:14.616384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 26/48] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:14.639943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da419e2-4d67-4b0f-87a2-608cc340c4d6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:14.664558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:14.742454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da419e2-4d67-4b0f-87a2-608cc340c4d6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:14.777704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:14.778337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.778619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.778924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.788836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:14.794297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:14.795164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:14.795936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 23/49] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:14.801282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:14.897262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d5291-1c76-4ddc-8a07-805970a1559e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:14.924747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:14.930128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:14.931455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.936835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.936835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.936835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:14.936835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:14.936835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706aeee-0b9a-4152-975b-c24a5b8eacf5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:14.942217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 27/50] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:14.948659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:14.959439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5315-b36d-4bc6-815f-15e1ed6ebd86 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.959728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5315-b36d-4bc6-815f-15e1ed6ebd86 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.960005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5315-b36d-4bc6-815f-15e1ed6ebd86 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.961324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 28/51] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:14.966848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.967132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.967411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.985043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:14.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.030934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.049555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cd377-a336-4db5-8477-1f4d3ee1bf91 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.051952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 29/52] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:15.074882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.082913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.083541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.083820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.084092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.085300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.087438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.093609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 24/53] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:14 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:15.094385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.173872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddb540-9b7c-4679-bce8-92d6615d613b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.192512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad297bf0-aea8-4ef2-a849-a80819a1e42e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.195644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.207302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.211361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.211361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.211361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.211361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.211361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.213259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 30/54] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:15.218960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.227458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.228104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.228387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.228670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.229799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.231307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.232881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 25/55] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:15.238438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.256758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.318540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654503-cb0c-4b08-b88d-ac98c010d62b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.339811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d4cd-c231-4a15-941b-6616fe6e14ad None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.352786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.353389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.353670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.353944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.354581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.356087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bf1ea-2479-409c-89cc-8ebe9304ae43 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.372924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.373599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.373884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.374160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.374911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.domain.id=default, user.id=c800e45fdce14a59ae5bf0f84f3671cd) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:15.375221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.376660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc73fb-edf9-4f94-b98e-029c3cec8c4e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.388063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 26/56] 10.222.0.36 () {56 vars in 1322 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/role_assignments?user.id=c800e45fdce14a59ae5bf0f84f3671cd&scope.domain.id=default&role.id=7409e93d755b445b891c0195b28879f6 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:15.464884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 31/57] 10.222.0.36 () {60 vars in 1140 bytes} [Tue Aug 9 17:56:15 2022] POST /identity/v3/projects => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:17.455796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389310ad-1059-457a-9064-9991a5f8669b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.456306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389310ad-1059-457a-9064-9991a5f8669b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.456387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389310ad-1059-457a-9064-9991a5f8669b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.457787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 27/58] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:17.464290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034c614-51f8-49f9-9032-de21b939453a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.464290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034c614-51f8-49f9-9032-de21b939453a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.464710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034c614-51f8-49f9-9032-de21b939453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.481768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-8034c614-51f8-49f9-9032-de21b939453a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:17.482950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8034c614-51f8-49f9-9032-de21b939453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:17.516437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:17.547271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 32/59] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:17.571828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389310ad-1059-457a-9064-9991a5f8669b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.662933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:17.682339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389310ad-1059-457a-9064-9991a5f8669b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.715319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.715871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.716150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.716421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.717521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.719035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc30c57-2dde-42f8-91f7-3d1922ad6039 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.720593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 28/60] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:17.726325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034c614-51f8-49f9-9032-de21b939453a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.823007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034c614-51f8-49f9-9032-de21b939453a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.856092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.856682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.856962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.857236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.857864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.859337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aedd8d-3249-4a7c-90be-a585c8f64897 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.959519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 33/61] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:17 2022] POST /identity/v3/users => generated 275 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:20.966008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2205f3-ab24-4136-9cab-660bb4148602 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.966523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2205f3-ab24-4136-9cab-660bb4148602 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.966523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2205f3-ab24-4136-9cab-660bb4148602 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.968066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 29/62] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:20.986499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47c4e-a1a0-46d1-a488-f8a56b7bfe0b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.987327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47c4e-a1a0-46d1-a488-f8a56b7bfe0b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.987745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47c4e-a1a0-46d1-a488-f8a56b7bfe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.015699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6bc47c4e-a1a0-46d1-a488-f8a56b7bfe0b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.017162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6bc47c4e-a1a0-46d1-a488-f8a56b7bfe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:21.048347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.107637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 34/63] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:21.141117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2205f3-ab24-4136-9cab-660bb4148602 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.158512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d55e5-2977-40cc-abf6-abc9b65d6daf None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.158814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d55e5-2977-40cc-abf6-abc9b65d6daf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.159098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d55e5-2977-40cc-abf6-abc9b65d6daf None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.160563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 35/64] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:21.180379 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7e2f-efa0-4ed0-9ef8-db2b5aabec48 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.180926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7e2f-efa0-4ed0-9ef8-db2b5aabec48 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.181372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7e2f-efa0-4ed0-9ef8-db2b5aabec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.216976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-101c7e2f-efa0-4ed0-9ef8-db2b5aabec48 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.218080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-101c7e2f-efa0-4ed0-9ef8-db2b5aabec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:21.274731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2205f3-ab24-4136-9cab-660bb4148602 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.310020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.310653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.310958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.311230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.313987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 36/65] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:21.315591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c685c-64a3-47a5-b50a-b40da91c8e4f None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.316078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c685c-64a3-47a5-b50a-b40da91c8e4f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.316510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c685c-64a3-47a5-b50a-b40da91c8e4f None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.317945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 37/66] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:21.323954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.324409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.324821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.326113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:21.331004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:21.332003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:21.332931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:21.334759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 30/67] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:21.340581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.353693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.354526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.355371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:21.428727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 38/68] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:21.430594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.453209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.470576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026736cd-5720-45ec-84ac-6c6a6ac73c42 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.514910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.515550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.515831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.516350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.545859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:21.550888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:21.551695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:21.552535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:21.560734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9707c10-81e1-4dc8-aa9c-f3460e02ddb0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.567567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 31/69] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:21.568492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.614995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.615975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.616424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.616853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.618198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:21.619156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.620761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.636640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 39/70] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:21.646990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.663136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.710600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcc36e-befa-4ca8-9a28-2dc8fed4ba33 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.753625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.774991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.775347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f7fe-f153-408a-a790-6d1790a3b949 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.775659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.775940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.776211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.787308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:21.794600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:21.795491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 32/71] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:21.797206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.814831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.815443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.815709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.815977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.816720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:21.817018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.818438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ea2de-d63c-43e8-b36a-bb6622900c0f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.828348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 40/72] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:21.829613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb12d-153d-48ab-8760-ffdec98b12bc None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.829910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb12d-153d-48ab-8760-ffdec98b12bc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.830184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb12d-153d-48ab-8760-ffdec98b12bc None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.831421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 41/73] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:21.832117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb12d-153d-48ab-8760-ffdec98b12bc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.911197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceffee-521b-41f8-99d6-6f1a1bc6c066 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.940901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.947286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.948087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.948435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.948776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.952384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb12d-153d-48ab-8760-ffdec98b12bc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.964639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.966125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.972599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 33/74] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:21.972599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.992055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.993920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.994203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.994478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.995349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:21.995654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.997030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93b5e5-e3e8-4311-91e3-48e74aa6a21b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.007109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 42/75] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:22.008431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0a90-a690-43d7-aba2-50a51d5b387d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.008719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0a90-a690-43d7-aba2-50a51d5b387d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.009104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0a90-a690-43d7-aba2-50a51d5b387d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.015945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.026813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9cde0a90-a690-43d7-aba2-50a51d5b387d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:22.028005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9cde0a90-a690-43d7-aba2-50a51d5b387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.078728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd521f-43c2-4545-9d88-90a7deb50194 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.093382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 43/76] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:22.103258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf235-4c2a-470f-a319-958f5419e974 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.103258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf235-4c2a-470f-a319-958f5419e974 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.103258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf235-4c2a-470f-a319-958f5419e974 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.103258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 44/77] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:22.103258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bf235-4c2a-470f-a319-958f5419e974 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.113060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.113770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.114123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.114467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.115738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.117932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.119553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 34/78] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:22.120319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.176096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.205534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bf235-4c2a-470f-a319-958f5419e974 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.223026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15269bb-2f7e-4d30-88c1-cd7c83fb5629 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.239404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.240012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.240291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.240565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.243722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.256960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.257690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.257966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.259074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.260035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.261520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672da82-b934-4081-b3c0-db8da6c10ecc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.263125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 35/79] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:22.264233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.264510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.264780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.282870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:22.284383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.312208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.313657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.350668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 36/80] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:22.358861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.408142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.419547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 45/81] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:22.420394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.453479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.454185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100fbbf-5aa6-4e08-b017-64790db4f52c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.489482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.490110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.490393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.490673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.501315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:22.506033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:22.506932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:22.507738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 37/82] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:22.508984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.528408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca0328-594c-46fe-a78b-1a59043e53f8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.561943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.562553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.562866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.563144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.606961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72a3e8-9fe1-4115-beb5-6e20fe7ae2ca None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.623029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.632517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.634100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.643156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.643754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.644029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.644360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.666258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.711746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.713241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.732309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 46/83] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:22.735604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.812015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 38/84] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:22.812850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.831487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.835117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef18f7-efd1-4d84-a102-3341a9ef0fe0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.869502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.870130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.870409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.870680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.871812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.873275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.874856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 47/85] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:22.875695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.878232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.920688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53dd8-cc86-4034-9e42-d6bd34df34d9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.954942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.955567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.955839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.956107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.957207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.958645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.960301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 39/86] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:22.963594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.972961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b929-e4af-4c72-992d-78691bb5adc3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.006758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.007363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.007626 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.007905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.008566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:23.008867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.010250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.018211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 48/87] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.019028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.048976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.067185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05418c1f-9ec9-4f93-bd3d-0d520b8737eb None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.082011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.100992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.104857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.104857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.104857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.104857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.105408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.107062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 40/88] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.107806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.119828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad20e6-e5b4-4a0c-b252-9049c24ce6a9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.154000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.154718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.155032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.155304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.156422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.157882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.159547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 49/89] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.160302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.210359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2165ec6-a417-401c-b259-4a26cf308da4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.245856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.246458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.246775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.247054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.248138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.249568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.258891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117332a-3548-4bd1-84fe-9a6cdaf1b0e3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.287863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.293386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.293980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.294261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.294527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.295278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:23.295580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.297012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.302060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 50/90] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.302843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.346372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 41/91] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:23 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:23.347193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.364321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.448727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963dff-54f1-4f85-98d3-2bf82c0aa3ba None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.506845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b9fe-8839-4dbe-9868-f2a679c3b326 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.509334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.510844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.511318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.511606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.512971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.514694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.518229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 51/92] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.519353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.556080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.556703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.556983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.557253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.557974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=c800e45fdce14a59ae5bf0f84f3671cd) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:23.558279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.559822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.565560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 42/93] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/role_assignments?user.id=c800e45fdce14a59ae5bf0f84f3671cd&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.566370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.659147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.676277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.692431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddc2d-7be6-46da-b381-32140d83ce4f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.730639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.731298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.731576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.731846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.735185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352011c-0306-40b1-8a04-15f2d8a8ec57 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.786535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.787356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.787734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.788109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.788949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:23.789847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.792186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.799532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 52/94] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:23.825689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.827288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.910791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.944936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 43/95] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:23.952961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:24.077874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09d88-5b25-4ea4-8399-c0060195d8e2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:24.111308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:24.117287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:24.118432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.118731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.119004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] PATH_INFO: `/v3/projects/9a1cbfba3e704c5fa613d18d5435e047` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.123429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1cbfba3e704c5fa613d18d5435e047)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:24.124902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d241f-74e0-4988-8e79-f5e042da0a15 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:24.126835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 53/96] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/projects/9a1cbfba3e704c5fa613d18d5435e047 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:24.132600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:24.256173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbd297-09dc-4e57-84c3-a55f5e178e2b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:24.304792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:24.305455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.305735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.306002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.306742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=9a1cbfba3e704c5fa613d18d5435e047, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:24.307042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:24.308473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486d55-05b8-4f7d-b82c-daf17a8cfb94 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:24.317403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 44/97] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:24 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=9a1cbfba3e704c5fa613d18d5435e047&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:26.269732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd56b4-b8df-45b3-9d84-22243d505059 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.270583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd56b4-b8df-45b3-9d84-22243d505059 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.271042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd56b4-b8df-45b3-9d84-22243d505059 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.272536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 54/98] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:26.278587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142abc-c04f-460e-91db-dd6882b32e7f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.279374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142abc-c04f-460e-91db-dd6882b32e7f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.279793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142abc-c04f-460e-91db-dd6882b32e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.303801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-84142abc-c04f-460e-91db-dd6882b32e7f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:26.305764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84142abc-c04f-460e-91db-dd6882b32e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.333708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:26.385155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 45/99] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:26.423114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd56b4-b8df-45b3-9d84-22243d505059 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.434289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba2de-abc4-4ed9-ab8b-b5d80daafef0 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.434569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba2de-abc4-4ed9-ab8b-b5d80daafef0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.434869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba2de-abc4-4ed9-ab8b-b5d80daafef0 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.436414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 46/100] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:26.441574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f5df-fb28-4cea-853b-d6749de04152 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.441853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f5df-fb28-4cea-853b-d6749de04152 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.442127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f5df-fb28-4cea-853b-d6749de04152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.482033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cc71f5df-fb28-4cea-853b-d6749de04152 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:26.483747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cc71f5df-fb28-4cea-853b-d6749de04152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.620223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd56b4-b8df-45b3-9d84-22243d505059 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:26.637972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 47/101] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:26.664362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349b0c-95cb-4064-95b2-6a6588ecabfa None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.665169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349b0c-95cb-4064-95b2-6a6588ecabfa None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.665656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349b0c-95cb-4064-95b2-6a6588ecabfa None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.667317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 48/102] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:26.679697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349b0c-95cb-4064-95b2-6a6588ecabfa None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.680214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:26.680857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.681137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.681433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.693658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e48209ff-4cf8-414b-80fb-415b42d72b1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:26.699983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:26.700858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:26.701693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 55/103] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:26.702339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.702575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.703016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.722832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:26.724229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.792963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:26.814736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 56/104] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:26.821232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.864726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:26.876415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349b0c-95cb-4064-95b2-6a6588ecabfa None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:26.924342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:26.924992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.925277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.925554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.930570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d688291-aff1-42c3-888b-9e7112e2c09d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:26.936503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-b1911280-02cf-4f1b-aa86-8460b3ccd217 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:26.941273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:26.942096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:26.942881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 49/105] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:26.944436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204458-b58e-423e-8013-723a324ad283 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.944727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204458-b58e-423e-8013-723a324ad283 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.945006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204458-b58e-423e-8013-723a324ad283 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.946835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 50/106] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:26.947854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204458-b58e-423e-8013-723a324ad283 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.975521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:26.976159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.976531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.976900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.977569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:26.977869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:26.979296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62a67c-bb9d-4cff-92d0-d5076e8b874d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:26.987911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 57/107] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:26.989337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70c54-5a55-43d6-bf92-922604637821 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.989620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70c54-5a55-43d6-bf92-922604637821 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.989890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70c54-5a55-43d6-bf92-922604637821 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.991103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 58/108] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:26.991780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec70c54-5a55-43d6-bf92-922604637821 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.052842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.058633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204458-b58e-423e-8013-723a324ad283 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.092632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.093260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.093542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.093817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.104432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:27.105331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec70c54-5a55-43d6-bf92-922604637821 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ff939602-604c-4733-86db-a7ff09251d52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:27.109108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:27.109958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:27.111152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 51/109] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:27.116174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb41f2c-7e7d-473d-bfff-69372a9a4a65 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.116469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb41f2c-7e7d-473d-bfff-69372a9a4a65 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.116750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb41f2c-7e7d-473d-bfff-69372a9a4a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.135357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8fb41f2c-7e7d-473d-bfff-69372a9a4a65 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:27.136461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8fb41f2c-7e7d-473d-bfff-69372a9a4a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:27.141904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.142838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.143368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.144131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.144848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:27.145150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.146560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.147714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.161915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 59/110] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.161915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.212593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 52/111] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:27.213622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.213907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.214180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.241387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:27.242919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:27.273010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757cdc-be5e-4387-9231-9acc1d1817df None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.284080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.307118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.307755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.308034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.308309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] PATH_INFO: `/v3/users/445b399252eb42c0b7b4de73b41b0079` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.315109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 53/112] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:27.317792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.323406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] RBAC: Authorizing `identity:get_user(user_id=445b399252eb42c0b7b4de73b41b0079)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.324874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 60/113] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/users/445b399252eb42c0b7b4de73b41b0079 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.327317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.391899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.417222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460a807-c2de-4fbf-93ff-83359639d171 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.426672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa927441-794c-4398-aeec-57e1b58b1f96 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.451625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.452234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.452517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.452780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.453432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:27.453730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.455142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.461146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.462633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.462955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.463233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.465320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.466786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986af0de-2910-4566-8df4-2f0df0515989 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.466999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 54/114] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.467846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 61/115] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0ce9a-52b4-4d0a-835b-b51dbc7bdb70 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0ce9a-52b4-4d0a-835b-b51dbc7bdb70 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0ce9a-52b4-4d0a-835b-b51dbc7bdb70 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 62/116] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:27.477926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0ce9a-52b4-4d0a-835b-b51dbc7bdb70 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.513684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.568901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29148-aec7-4f25-8f41-fad00344b8cc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.578310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0ce9a-52b4-4d0a-835b-b51dbc7bdb70 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.602567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.603223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.603510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.603787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.604896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.606343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.607983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 55/117] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.608697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.612426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.616416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.617586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.617968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.618240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.684588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.685981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.708827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd0797-61c9-4a95-8c20-30c44f1934ed None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.719713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.742394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.743255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.743541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.743817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.753706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:27.760048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:27.760888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:27.761630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 56/118] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:27.762366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.789815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 63/119] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:27.789815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.827712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.864417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefbad9-1f09-4ff1-98bc-f0ad22621c2f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.888135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714012-f044-487a-a807-f2ddaf8d8cfc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.898969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.899672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.899954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.900223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.901350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.902822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca33a8-119d-4cfd-93a3-658d31bab3b4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.904403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 57/120] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.905487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.905767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.906042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.923263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:27.923418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.924001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.924285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.924448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:27.924561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.925669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.927164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.928788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 64/121] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:27.929574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:27.933020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.995396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 58/122] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:27.996449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.041686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fba2d3-b663-4454-bdc1-43e3ef82b315 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.059009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.075852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.076457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.076736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.077006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.077679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.079166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.100708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131efcc-beba-41bd-8d7e-1f0aabc9d07f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.134664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.135304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.138741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.138741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.138741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:28.138741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.138741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.145757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 59/123] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.146597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.169811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 65/124] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:56:27 2022] POST /identity/v3/projects => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:28.169811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.227691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.267113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bc88-31c4-4167-b087-2dcf50cee062 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.275836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ffda-d908-49aa-a6b7-9b61a5a0edda None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.292476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.303243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.303953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.304244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.305041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.306279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.307845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.314385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 60/125] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.314385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.324852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.325496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.326959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.326959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.328062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.329522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.331178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 66/126] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.331932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.439674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b8f5-7360-436c-9ba3-ac8f204c014a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.453061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cbf04-6724-4b83-8d3d-3faafbf80f1a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.479359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.480182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.480471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.480913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.481083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.483378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, user_id=a5de9c855767447ba06e66c6b1bf1a3e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.485456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.491169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.491806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.492139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.492413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.502986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:28.507840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:28.508728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:28.509658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 67/127] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:28.510437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.571942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 61/128] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:28 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:28.572789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.637479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.647502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877adbc-d203-408f-bde6-df942f3c3cd7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.691030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.691674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.691954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.692228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.693596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.695639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.697438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 68/129] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.698209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.703500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f8b66-e862-4f26-8c06-0cba8c96c199 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.754305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.756249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.756249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.756249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.756886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=445b399252eb42c0b7b4de73b41b0079) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:28.759295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.759498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.761282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.766206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 62/130] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/role_assignments?user.id=445b399252eb42c0b7b4de73b41b0079&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.766915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:28.821901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352dc60-a33a-4508-8a5d-05f3bd40db27 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.865714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.866311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.866588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.867090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/users/c800e45fdce14a59ae5bf0f84f3671cd/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.870055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, user_id=c800e45fdce14a59ae5bf0f84f3671cd, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.871727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.878750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3b732-f101-4833-8eff-2e05ec44d2da None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:28.899434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.918616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:28.919281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.919566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.919844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.920519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:28.920844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:28.922362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:28.932876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 63/131] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:28.933754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.003522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 69/132] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:28 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/users/c800e45fdce14a59ae5bf0f84f3671cd/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:29.004537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.048486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a84cc-f626-4e6e-a660-a9949c03a80a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.095132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.096105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.096418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.096763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.098939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.100715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.104317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 64/133] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:28 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:29.110310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.212159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:29.221783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00505a49-d76a-4e2b-acb2-16c27698058b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.239473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:29.267629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.268646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.269187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.269661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.275086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.277363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.279105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 70/134] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:29 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:29.308243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.314220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea351cd-aad1-4ac7-aabe-a900a541ece5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.371742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.372224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.372845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.375288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, user_id=a5de9c855767447ba06e66c6b1bf1a3e, role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.377363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.437678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e4e9-8976-408c-97ab-66ba74116e30 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.446613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 65/135] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:29 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:29.488537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.489626 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.490176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.490725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] PATH_INFO: `/v3/projects/9a1cbfba3e704c5fa613d18d5435e047` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.492802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1cbfba3e704c5fa613d18d5435e047)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.494683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d03478-36aa-4e13-bb38-df3361439318 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.497106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 71/136] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:29 2022] GET /identity/v3/projects/9a1cbfba3e704c5fa613d18d5435e047 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:29.502922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.644622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a215a1-e997-4084-a903-dd1788ba07b5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.657394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:29.695049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.696286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.697631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.698126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] PATH_INFO: `/v3/projects/9a1cbfba3e704c5fa613d18d5435e047/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.700558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a1cbfba3e704c5fa613d18d5435e047, user_id=a5de9c855767447ba06e66c6b1bf1a3e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.702335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eee63-4636-473c-955b-4e257adf5991 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.817161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 66/137] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:29 2022] PUT /identity/v3/projects/9a1cbfba3e704c5fa613d18d5435e047/users/a5de9c855767447ba06e66c6b1bf1a3e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:30.650530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b194ce7-e848-4519-82a4-0e13e1eacfc7 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.651581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b194ce7-e848-4519-82a4-0e13e1eacfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.652073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b194ce7-e848-4519-82a4-0e13e1eacfc7 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.653609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 72/138] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:30.659101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.659924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.660396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.679829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:30.681350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:30.768573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 67/139] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:30.806417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b194ce7-e848-4519-82a4-0e13e1eacfc7 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.870361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:30.936175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b194ce7-e848-4519-82a4-0e13e1eacfc7 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.987452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.988116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.988405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.988690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.989861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.991412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53350a5-c622-4eae-9c50-daf6f330b86e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.993951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 73/140] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:30.999803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.126571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:31.133500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a163703-cd08-4d32-9704-4fec9548e21f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.168391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.169108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.169390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.169655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.171814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.173290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a80b5-202c-4eff-bde1-a4024809e782 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.268977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 68/141] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:31.452501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb5d09-21b0-4e1b-bcbb-1fa5825bea67 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.452871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb5d09-21b0-4e1b-bcbb-1fa5825bea67 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.452971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb5d09-21b0-4e1b-bcbb-1fa5825bea67 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.456119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 74/142] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:31.468399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.468684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.468961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.495380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:31.496578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:31.527627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf8a1-739c-4a3b-9b43-3e1b55b4ae64 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.527933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf8a1-739c-4a3b-9b43-3e1b55b4ae64 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.528181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf8a1-739c-4a3b-9b43-3e1b55b4ae64 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.529553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 75/143] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:31.544608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.544902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.545162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.582412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 69/144] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:31.587696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:31.588917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:31.606193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.691949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:31.708918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 76/145] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:31.711002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5d88b-9a53-4aab-80fb-16b5e4b27b1a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.745081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.751535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.752217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.752667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.753123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.766484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:31.778032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:31.780299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 70/146] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:31.782313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.879854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:31.894534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caab056-f3d4-49b5-8f3f-2975c76f501e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.936393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.937008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.937283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.937549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.938732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:31.939051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.940587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.955211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 71/147] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:31.958116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.962888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:31.994639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f3f81-7532-4004-8a9a-1a42a5881516 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.046399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.047150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.047392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.047731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.063860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:32.065684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8a28-1ab4-4274-b5bb-dcaeeaaf4469 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:32.068623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:32.069491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:32.070593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 77/148] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:32.078375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.104822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.105451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.105723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.105986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] PATH_INFO: `/v3/users/445b399252eb42c0b7b4de73b41b0079` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.107155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] RBAC: Authorizing `identity:get_user(user_id=445b399252eb42c0b7b4de73b41b0079)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.108604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.115651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 72/149] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/users/445b399252eb42c0b7b4de73b41b0079 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:32.116504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.190270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.223503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8968e2-9f85-44e6-87fc-a6617854cbbe None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.260219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.260829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.261102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.261372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.262507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.263989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.265795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 73/150] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:32.271497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.338091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521686f4-2b52-4a21-a81b-f85dd7aa78ec None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.348646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.380748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6afad0-8d52-4085-9629-679529708984 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.380957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.381624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.381905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.382176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.383752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:32.384163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.386529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.395976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 78/151] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:32.402656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.426154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.427408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.427686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.427958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/users/445b399252eb42c0b7b4de73b41b0079/roles/e4d598d28bdb4541aebb9197eb7e37f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.433199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, user_id=445b399252eb42c0b7b4de73b41b0079, role_id=e4d598d28bdb4541aebb9197eb7e37f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.435732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641f5d-d51d-4254-8891-d1c47b9b9a74 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.512126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.518766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 74/152] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:32 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/users/445b399252eb42c0b7b4de73b41b0079/roles/e4d598d28bdb4541aebb9197eb7e37f6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:32.520119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bb85-1562-4f96-8f76-ecf3656dfd60 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.520397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bb85-1562-4f96-8f76-ecf3656dfd60 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.520716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bb85-1562-4f96-8f76-ecf3656dfd60 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.525487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 75/153] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:32.530109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.530378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.530649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.558516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e7e3d-d97d-469d-a373-45952e46c26e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.579778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:32.581078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:32.609414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.610072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.610450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.610750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.614417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.616231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.618378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 79/154] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:32.624444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.687121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.753938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 76/155] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:32.794837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791b383-90a9-42f3-b401-d43cad5d6532 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.829811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.845285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.845899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.846176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.846445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.949324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.951236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d36eba-da8c-4384-a785-02d24ece17dc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.971539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aa4a6-23a8-4658-a083-462e1bb95099 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.007149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.015070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.022228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.024515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.024806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.025079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.041111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c5df37e6-4839-4728-9a0c-f23e29914b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:33.048126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:33.049055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:33.049823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 77/156] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:33.051485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0ed1-71cd-466e-a2a3-02aedba2c4f3 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.051789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0ed1-71cd-466e-a2a3-02aedba2c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.052120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0ed1-71cd-466e-a2a3-02aedba2c4f3 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.053362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 78/157] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:33.055472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0ed1-71cd-466e-a2a3-02aedba2c4f3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.091203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 80/158] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:56:33.092705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.098731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.099316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.126729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:33.128268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:33.211732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0ed1-71cd-466e-a2a3-02aedba2c4f3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.252961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 81/159] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:33.253955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.291546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.292076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.292350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.292767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.293444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:33.293740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.295217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.312330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 79/160] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:33.313182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.360239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.415733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.425269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bfcf3-8866-409c-85ca-b241cd565352 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.499265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fede-ee1d-4e9f-a973-ecf037a9460a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.537639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.539330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.539821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.540393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.554127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.554127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.554127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.554127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.554648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:33.555449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.556924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-e9d2251e-2fe6-484b-8d2f-a2b2ed40a85f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:33.557962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:33.559260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:33.560363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:33.561212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 82/161] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:33.561279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 80/162] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:33.561342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.561483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee34bc-44c4-4d08-bc22-967c3da40121 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.561483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee34bc-44c4-4d08-bc22-967c3da40121 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.561483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee34bc-44c4-4d08-bc22-967c3da40121 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.562036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 81/163] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:33.567460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050625e-9070-459d-8321-8eecb78377bb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.568678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050625e-9070-459d-8321-8eecb78377bb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.568960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050625e-9070-459d-8321-8eecb78377bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.597947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7050625e-9070-459d-8321-8eecb78377bb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:33.599185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7050625e-9070-459d-8321-8eecb78377bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:33.676751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 82/164] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:33.677524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050625e-9070-459d-8321-8eecb78377bb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.752698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d22c0-39ec-4007-9bf7-b97391d80dc3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.758684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.789261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.790325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.790793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.791218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] PATH_INFO: `/v3/users/c800e45fdce14a59ae5bf0f84f3671cd` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.792547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] RBAC: Authorizing `identity:get_user(user_id=c800e45fdce14a59ae5bf0f84f3671cd)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.794216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.796139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 83/165] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/users/c800e45fdce14a59ae5bf0f84f3671cd => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:33.797623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.805563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:33.811465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050625e-9070-459d-8321-8eecb78377bb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.847663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.848408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.848997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.849463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.850398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:33.851502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.853403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.860209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 83/166] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:33.861262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.912858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b75b71-809b-4241-931d-890c7291fb0e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.949213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.950178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.950639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.951188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.952064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:33.952579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.954156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.964266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 84/167] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:33.965492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.982038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620633-2e83-41fc-b6f7-8d1dea795f12 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.991078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.039309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.040228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.040504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.040768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.057183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:34.072000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:34.072671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:34.073530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 84/168] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:34.074935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.124049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b16717-c412-43b5-9d49-c0ce92e2ade2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:34.164251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.164882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.165159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.167018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.277343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:34.284356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:34.303151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762813d-ea2a-4efa-af59-af9572e2ea45 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:34.406363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.406957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.407241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.407510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.408735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:34.410259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:34.412106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 85/169] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:34.412666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.421320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.518488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.537756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 85/170] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:34.541530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.634870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8fafc-e4f5-486c-9ef1-bca28fdeb598 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:34.672360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.672884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.673159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.673429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.674088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:34.674386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:34.676244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:34.685171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 86/171] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:34.685990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.851679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.860327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531160-a10b-4f6d-b8e6-1cfca5847055 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:34.914688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.915348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.915625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.915894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.919389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:34.920848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:34.922553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 87/172] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:34.923278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.935617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124ff77-9dbb-4a56-8000-b4946d76c2a3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.008845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.009514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.009800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.010079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.056162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2bad5-801c-4433-920d-b056aa7c1850 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.075094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.091542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.092172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.092451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.092716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] PATH_INFO: `/v3/users/a5de9c855767447ba06e66c6b1bf1a3e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.093847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] RBAC: Authorizing `identity:get_user(user_id=a5de9c855767447ba06e66c6b1bf1a3e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.095429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.097022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 88/173] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/users/a5de9c855767447ba06e66c6b1bf1a3e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:35.097784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.135091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.167595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.169102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.206534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f750c-42de-43e9-ac8e-fd97af1a0986 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.244000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.244887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.245265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.245633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.246574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=c800e45fdce14a59ae5bf0f84f3671cd) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:35.247006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.248720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.255346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 89/174] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:35 2022] GET /identity/v3/role_assignments?user.id=c800e45fdce14a59ae5bf0f84f3671cd&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:35.256173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.341436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 86/175] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 803 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:35.354527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.414315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.428025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d1909-aacf-43d7-83be-a83c4f6c8c73 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.503629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.533818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.534442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.534733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.535101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.592332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a01029-f71f-4ab6-bed6-760751e92792 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.660252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.660888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.661165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.661434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.662736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.664818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.671118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 87/176] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:35 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:35.672739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.681154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.682962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2797986-4f22-4e45-8cff-c1069dc78ed1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.797597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.801311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 90/177] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:35.802324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778315-39d4-4a75-b795-ed2cb2cb7e88 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.802602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778315-39d4-4a75-b795-ed2cb2cb7e88 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.807057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778315-39d4-4a75-b795-ed2cb2cb7e88 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.808652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 91/178] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:35.810767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03778315-39d4-4a75-b795-ed2cb2cb7e88 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.849008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c24b5-629e-4878-9c23-e6e37c6b59fc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.895006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:35.902547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.903199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.903641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.904053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.904997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:35.905481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.911390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e2463-232a-4727-abaf-64eb471f682e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.918209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 88/179] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:35 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:35.919495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba76cc2-efbf-4926-af25-4660b21af1c2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.919779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba76cc2-efbf-4926-af25-4660b21af1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.920055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba76cc2-efbf-4926-af25-4660b21af1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.929445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03778315-39d4-4a75-b795-ed2cb2cb7e88 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.954985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-dba76cc2-efbf-4926-af25-4660b21af1c2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:35.956213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-dba76cc2-efbf-4926-af25-4660b21af1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:35.993047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.994013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.994293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.994559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] PATH_INFO: `/v3/projects/9a1cbfba3e704c5fa613d18d5435e047` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.995736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1cbfba3e704c5fa613d18d5435e047)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.997188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f6d-f7a8-417a-82d4-4459a9317888 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.999115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 92/180] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:35 2022] GET /identity/v3/projects/9a1cbfba3e704c5fa613d18d5435e047 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.000300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b668fb-f224-4620-9df6-a5bfb817600f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.000579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b668fb-f224-4620-9df6-a5bfb817600f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.000848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b668fb-f224-4620-9df6-a5bfb817600f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.007967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 93/181] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:36.009124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b668fb-f224-4620-9df6-a5bfb817600f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.087543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.118127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 89/182] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:36.119252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bea8e-3709-4a5f-9f41-6de649677308 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.119535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bea8e-3709-4a5f-9f41-6de649677308 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.119794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bea8e-3709-4a5f-9f41-6de649677308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.128676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b668fb-f224-4620-9df6-a5bfb817600f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.165570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.166187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.166462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.166757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.167502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=9a1cbfba3e704c5fa613d18d5435e047, user.id=a5de9c855767447ba06e66c6b1bf1a3e) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:36.167792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-325bea8e-3709-4a5f-9f41-6de649677308 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.167895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.168998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-325bea8e-3709-4a5f-9f41-6de649677308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.169237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.176105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 94/183] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/role_assignments?user.id=a5de9c855767447ba06e66c6b1bf1a3e&scope.project.id=9a1cbfba3e704c5fa613d18d5435e047&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.180292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.306983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 90/184] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:36.308804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d12ca-aa4a-4898-88a5-0487ba17b85b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.309295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d12ca-aa4a-4898-88a5-0487ba17b85b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.309652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cb727-0108-48c3-8d4e-a85190fe0327 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.309966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d12ca-aa4a-4898-88a5-0487ba17b85b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.311611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 91/185] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:36.316420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5f560-cb3e-48ce-881e-1a99e5946439 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.316945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5f560-cb3e-48ce-881e-1a99e5946439 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.317405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5f560-cb3e-48ce-881e-1a99e5946439 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.318994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 92/186] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:36.323275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.323838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.324322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.345810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.347393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.347731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.360579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.361513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.361811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.362137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.375460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-863ef961-a920-4676-a0aa-48d37408036b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:36.380638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:36.381524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:36.383173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 95/187] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:36.383817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.383881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.384449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.407111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.408316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.413172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.432180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 93/188] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:36.433300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.496345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 96/189] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:36.497192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.540395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0a16-9120-4b9e-b5f0-d64f1f597ed3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.573647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.574268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.574543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.574887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.585590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:36.590259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:36.591261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:36.592062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 94/190] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:36.592713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.604442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab1ea-93d7-4f9c-af99-90c19b3b136e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.622941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.640608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.641209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.641486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.641753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.642190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.642403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:36.642715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.644119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.653696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 97/191] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.653696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.701713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e8be4-5a2b-40e1-b234-483d2bb64122 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.738048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.738671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.738976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.739242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.749323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:36.753982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:36.755007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:36.755806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:36.756414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58ab8c-48d5-45c0-94af-816f626ee80c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.756485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 95/192] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:36.756485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.790183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.790907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.791188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.791461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.792567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.794006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.795818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 98/193] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.796425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.844345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.858219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334940b-3e4c-4a17-8b06-82740c298a6b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.865484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.893177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.893782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.894102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.894378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.895036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ed7e7-a896-432c-8c87-4113b1948efc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.895164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:36.895419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.896800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.905372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 96/194] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.906158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.934374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.935034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.935312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.935582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.948848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.950306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.951949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 99/195] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:36.952693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.008238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8471-4811-4694-86b9-ee42e937e85c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.041298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.041894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.042167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.042434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.043153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:37.043450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.044812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.051549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabfcbd-d708-4aa4-b171-13b2cf0efddd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.052485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 97/196] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:37.053257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.064493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.082900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.091545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.092145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.092429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.092693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.093327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.094799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.152623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448617f2-9853-44f9-96e5-38355ca3b6f0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.184366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.184983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.186014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.186014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.186746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.188199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.189755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 98/197] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:37.190507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.216960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 100/198] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:37.227978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.300228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.305660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedb34-69c5-407e-b1ae-dc57326a78b9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.342856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.343469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.343742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.344006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.347303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cc0c2-592a-46e2-b8df-44462fb673f1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.385825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.386456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.386811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.387081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] PATH_INFO: `/v3/users/445b399252eb42c0b7b4de73b41b0079` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.388209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] RBAC: Authorizing `identity:get_user(user_id=445b399252eb42c0b7b4de73b41b0079)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.389658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.392049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 101/199] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/users/445b399252eb42c0b7b4de73b41b0079 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:37.396526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.423181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.424895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.440520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.541524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f93ef-1333-4ab3-8b6c-351bdfdfd80e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.556572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.564989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 99/200] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:37.565817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.593407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.594813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.595592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.596060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.681135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815151f-e10d-4e20-bb64-afadb348c3e4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.695454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.696954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.729689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.731724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.732718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.733192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.733674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.811680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.815740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.817351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.826849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 102/201] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:37.828062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.928652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 100/202] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:37.933426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.946199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885161c4-b160-4b50-a4dc-9b5ae42d80ab None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.976599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.982399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.983319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.983796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.984291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.985681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.987349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.999442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 103/203] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:37.999442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.039943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:38.045213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77993-08f3-45f0-b8c8-bdf50ed081bb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.084743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.086308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.086795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.087284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.088448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.089926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.091649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 101/204] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:38.092411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.111149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc983-34ba-442d-8240-cf15f2427119 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.151049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.151765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.153378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.153666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.154973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.156490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.161988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 104/205] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:38.162934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.205744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfdb46-6fd8-4e5e-b2c6-96433e4158c0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.231433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:38.245762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.248234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.248234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.248234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.248569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=1fecb9b00c7e4d6295f7e5603a4607ea) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:38.249149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.251592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.258835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 102/206] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:38 2022] GET /identity/v3/role_assignments?user.id=1fecb9b00c7e4d6295f7e5603a4607ea&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:38.259821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.299397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:38.309789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faffa43-449e-4166-9713-f085f40afe5b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.390977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21330d12-dce0-4efe-8162-3e819351f317 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.434689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.435599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.436075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.436545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.437972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, scope.project.id=bc75ae0177fe43469516093786d30add, user.id=445b399252eb42c0b7b4de73b41b0079) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:38.438450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.440175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d5b31-e4c9-4e0d-96da-c187f50b9c3d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.448391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 103/207] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:38 2022] GET /identity/v3/role_assignments?user.id=445b399252eb42c0b7b4de73b41b0079&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:38.453810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.454726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.455181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.455606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.456639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=1fecb9b00c7e4d6295f7e5603a4607ea) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:38.457143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.459011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750a86f-8786-462e-a4fe-5a8353bf1f5c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.466744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 105/208] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:38 2022] GET /identity/v3/role_assignments?user.id=1fecb9b00c7e4d6295f7e5603a4607ea&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:39.465555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca972c-e79f-4727-8f02-dbb82987f878 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.466004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca972c-e79f-4727-8f02-dbb82987f878 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.466290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca972c-e79f-4727-8f02-dbb82987f878 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.467997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 104/209] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:39.473389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564434b-687c-4799-b20c-92f22b00e3d9 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.473791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564434b-687c-4799-b20c-92f22b00e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564434b-687c-4799-b20c-92f22b00e3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.493451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a564434b-687c-4799-b20c-92f22b00e3d9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:39.494745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a564434b-687c-4799-b20c-92f22b00e3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:39.567002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 106/210] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:39.581560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5989eb-3bc8-498e-8523-c9b5b0e4088a None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.582183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5989eb-3bc8-498e-8523-c9b5b0e4088a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.582619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5989eb-3bc8-498e-8523-c9b5b0e4088a None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.584343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 105/211] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:39.590433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.590983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.592262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5989eb-3bc8-498e-8523-c9b5b0e4088a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.593424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.617875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:39.619123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:39.690596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:39.704816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 107/212] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:39.717367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:39.727292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5989eb-3bc8-498e-8523-c9b5b0e4088a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.735118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.781116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.782494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.782954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.783777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.799087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:39.806288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:39.807371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:39.808368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:39.811200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 106/213] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:39.817013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.845600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc45c50-641b-4942-b2d1-c8e4e5e3141b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.882487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.883234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.883513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.883780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.884898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:39.886339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:39.892700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 108/214] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:39.894080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.942187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5fc0-60da-4b30-ba50-559553172c1a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.956396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:39.989899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.991224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.992563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.992982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.993951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:39.994439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:39.996655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.003784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073640c-2bbb-4ba0-a9d9-c1ff317cbac3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.008173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 107/215] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:40.013264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.036417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.043591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.048831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.048831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.048831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.048831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.048831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.125077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d52d-ffea-4b15-bdb9-930d2998dc08 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.143461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 109/216] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:40.162697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.164103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.164427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.164715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] PATH_INFO: `/v3/users/49d0820b3d0f48efbf18820a3671996b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.182328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] RBAC: Authorizing `identity:get_user(user_id=49d0820b3d0f48efbf18820a3671996b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.183856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.185896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 108/217] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/users/49d0820b3d0f48efbf18820a3671996b => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:40.191628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.305850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe034-fe0b-4fb0-aad8-3179daffe355 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.322014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.343887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.346860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.346860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.346860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.419581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.421112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc738-51f0-403d-9444-d19407dfefd6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.531628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 110/218] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:40.537633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.632992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.661959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9c274-a2a2-41a7-b79a-ef0c6f212740 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.685184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2c71-9d6c-4a3b-a12e-7dc78d8e425d None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.685483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2c71-9d6c-4a3b-a12e-7dc78d8e425d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.685757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2c71-9d6c-4a3b-a12e-7dc78d8e425d None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.687386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 111/219] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:40.692254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3813f-01c3-41bb-8ad9-8b10a90b11de None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.692539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3813f-01c3-41bb-8ad9-8b10a90b11de None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.692816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3813f-01c3-41bb-8ad9-8b10a90b11de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.700566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.701451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.701899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.702337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.703702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.705489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957eec-c742-4290-bed0-2b488b23c1bb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.707182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 109/220] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:40.708429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8e1d-9964-4364-8089-866a267d0ed9 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.708709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8e1d-9964-4364-8089-866a267d0ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.708975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8e1d-9964-4364-8089-866a267d0ed9 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.710854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 110/221] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:40.714917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-03a3813f-01c3-41bb-8ad9-8b10a90b11de None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:40.715559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a8e1d-9964-4364-8089-866a267d0ed9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.716283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-03a3813f-01c3-41bb-8ad9-8b10a90b11de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.727313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.802466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 112/222] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:40.804641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.805187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.805731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.824533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:40.826094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.853953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a8e1d-9964-4364-8089-866a267d0ed9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.891147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.891878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.891987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.892163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.892988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=49d0820b3d0f48efbf18820a3671996b) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:40.893288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.894817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.901056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 111/223] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/role_assignments?user.id=49d0820b3d0f48efbf18820a3671996b&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:40.901847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.912122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 113/224] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:40.924727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.936094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.032082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.075137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0839654-e697-4b34-a5d1-ed5e98ff048c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.120391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a07c6-c6a9-45e8-87f7-9ab0c6a12b3f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.136381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.137019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.137296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.137566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.150086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:41.156105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:41.157038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:41.157822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 114/225] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:41.163765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.168787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.169565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.169928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.170288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.195903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:41.201632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:41.202481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:41.203314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 112/226] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:41.208946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.270907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd89-68d1-44c4-a36a-73694d64de63 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.308154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.308767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.309042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.309312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.309977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:41.310274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.311684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.320532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 115/227] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:41.321002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.326233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.367500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.433324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ba30-ca1f-4f70-967e-32e7205217d1 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.442486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784888a7-7067-4b30-ad73-29db2a99b622 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.477490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.478255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.478532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.478826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.479949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.481383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.487484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 116/228] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:41.488659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.515519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.516124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.516394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.516660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.517419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:41.517812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.519209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.539219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 113/229] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:41.545067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.554385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.636941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cbcab-7ba8-4c0d-9253-89f1e080f980 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.704775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd49e-5ad8-4daf-a3f4-8aeff34b15a9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.708352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.708996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.709290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.709531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.715613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.717137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.718964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 117/230] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:41.727549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.756531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.762593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.763257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.763559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.769472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.769472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.769472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.769472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 114/231] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:41.779146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.921163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287be1d-3c0e-4bd7-8ff5-c5d62a051ae1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.961799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fdcf2-1852-49a8-a00b-234ab380b748 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.979879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.990396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.991160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.991522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.991790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/1fecb9b00c7e4d6295f7e5603a4607ea/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.997183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, user_id=1fecb9b00c7e4d6295f7e5603a4607ea, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.998636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f425da2-8db1-40f5-8c06-0e03fa442252 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:42.016000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:42.016670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.016941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.017203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.018355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:42.019850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:42.021893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 115/232] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:41 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:42.028352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:42.096274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 118/233] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:41 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/1fecb9b00c7e4d6295f7e5603a4607ea/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:42.098730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:42.142037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b74933-462b-4305-9cf4-cfdebcb1deee None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:42.181381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:42.182047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.182326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.182587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/1fecb9b00c7e4d6295f7e5603a4607ea/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.184929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, user_id=1fecb9b00c7e4d6295f7e5603a4607ea, role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:42.186422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfd98f-9b87-46f8-a955-7d67c2c25d9f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:42.302333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 116/234] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:42 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/1fecb9b00c7e4d6295f7e5603a4607ea/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:42.775048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516fc3-9d09-425a-946f-61a6c97ad9bb None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.775566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516fc3-9d09-425a-946f-61a6c97ad9bb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.775566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516fc3-9d09-425a-946f-61a6c97ad9bb None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.777179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 119/235] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:42.784425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.784850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.785021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.804080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:42.805330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:42.812307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b931a-b864-4c8b-9f68-f4d335838f9b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.812582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b931a-b864-4c8b-9f68-f4d335838f9b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.812851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b931a-b864-4c8b-9f68-f4d335838f9b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.814314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 120/236] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:42.820365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:42.821018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:42.821498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:42.859716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:42.861262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:42.881396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 117/237] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:42.906604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:42.965275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:42.995436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 121/238] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:43.018652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.025046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b75386-fff5-4d18-872e-190d8b3923ca None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.072836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.073828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.074291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.074764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.077097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.078769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.081673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 118/239] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:43.090914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.190086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c023a3-b5eb-4fa7-be2c-c4da9a4c8ec2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.220539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:43.227287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.228400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.228872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.229331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.240790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:43.247036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:43.247910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:43.248705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:43.251272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 122/240] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:43.265673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.325534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99de9b-b8d2-432c-ad82-7d35c5bd4dd3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.371103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:43.389582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.390185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.390456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.390732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.391388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.392834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b61222-365c-4e64-8c2b-93e1bff8e561 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.431108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc809-4257-4051-b43e-19cbf4290e12 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.471658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.472384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.472729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.473070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.473895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:43.474283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.475189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 119/241] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:43 2022] POST /identity/v3/users => generated 293 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:43.476573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.477851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da3e9e-5861-4e0a-8e06-8b64e8bc3148 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.478648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da3e9e-5861-4e0a-8e06-8b64e8bc3148 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.479166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da3e9e-5861-4e0a-8e06-8b64e8bc3148 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.481209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 120/242] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:43.489607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.490215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.490792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.491961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 123/243] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:43.500284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.515131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:43.516091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:43.561541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:43.594920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 121/244] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:43.614324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.627602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542486b-5fc4-40f2-b09e-2417e64aa8af None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.669423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.670390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.670945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.671442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] PATH_INFO: `/v3/users/de8adf41f94e44098aa4ad45bf6a196f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.690896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] RBAC: Authorizing `identity:get_user(user_id=de8adf41f94e44098aa4ad45bf6a196f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.692760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.695517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 124/245] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/users/de8adf41f94e44098aa4ad45bf6a196f => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:43.701395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.734254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12c058-abcf-47f5-a17f-4e98ce11d3ab None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.745398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:43.773415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.774042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.774313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.774578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.788429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:43.793150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:43.794178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:43.795242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:43.803314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 122/246] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:43.807491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.827852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3d61-f5e3-47d3-ab6a-dd490b797465 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.830360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:43.871188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.872182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.872594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.873044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.920109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320d9-f715-40d2-ac20-362d58c08ff2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:43.949988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.951716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.958315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:43.959065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.959440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.960497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.961433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:43.961832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:43.963450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:43.978104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 123/247] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:43.980319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.013484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.078004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 125/248] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:56:44.083840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.087288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.093861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9fc4c-701c-4019-a2aa-6ef90bea3067 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.130852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.131614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.131992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.132353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] PATH_INFO: `/v3/users/49d0820b3d0f48efbf18820a3671996b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.133777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] RBAC: Authorizing `identity:get_user(user_id=49d0820b3d0f48efbf18820a3671996b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.135481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.141724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 124/249] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/users/49d0820b3d0f48efbf18820a3671996b => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:44.143030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.199446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6111b-dd10-4fef-a127-bb1a3191862e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.238932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.239893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.240340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.240829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.242735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.244520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.246929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 126/250] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:44 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:44.248439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.252232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.253962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cc15f-577f-494f-956c-2f08d50555ce None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.295234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.295881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.296155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.296425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.297546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.299155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.300905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 125/251] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:44 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:44.307697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.345765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.388917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395974d-0ea8-4114-9f70-f7bc46948861 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.432053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebfb02-01e2-40d6-9169-3e55fed507ef None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.437278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.437991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.438360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.438747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.439571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=de8adf41f94e44098aa4ad45bf6a196f) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:44.439991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.442084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099dffc-f095-4046-a465-ee06663a88be None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.450820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 127/252] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:44 2022] GET /identity/v3/role_assignments?user.id=de8adf41f94e44098aa4ad45bf6a196f&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:44.480119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.481049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.481468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.481877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/49d0820b3d0f48efbf18820a3671996b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.484113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, user_id=49d0820b3d0f48efbf18820a3671996b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.485879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0aa336-626b-4a5c-86a8-a4f7931b7d9c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.565802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 126/253] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:44 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/49d0820b3d0f48efbf18820a3671996b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:44.776640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edbf97-0dda-44ef-915b-f8fd72dc44ca None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.777272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edbf97-0dda-44ef-915b-f8fd72dc44ca None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.777771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edbf97-0dda-44ef-915b-f8fd72dc44ca None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.779339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 128/254] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:44.784877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.785163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.785446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.813616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:44.814848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.841736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.904859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 127/255] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:44.928196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edbf97-0dda-44ef-915b-f8fd72dc44ca None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.055542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edbf97-0dda-44ef-915b-f8fd72dc44ca None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.096367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.096999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.097274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.097536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.104405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:45.114520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4b05f110-6b78-4ade-a92c-81cf9c8aab86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:45.121558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:45.122422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:45.123253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:45.126995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 129/256] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:45.130126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.134223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3aa0d-76b5-42d4-828f-900d751807f1 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.134600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3aa0d-76b5-42d4-828f-900d751807f1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.135014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3aa0d-76b5-42d4-828f-900d751807f1 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.137330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 130/257] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:45.142925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.143306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.143674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.172772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:45.173990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:45.318491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 131/258] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:45.346037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.358642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddbf14-0c0a-4ebb-97f1-67aaee162c11 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.443427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.444109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.444385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.444660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.445346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:45.445645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.447050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.472780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 128/259] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:45.479657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.519512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468359-08aa-4cf0-82bc-6a5d4c97a667 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.564999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:45.565929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:45.574843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.575473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.575747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.576015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.591714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:45.596463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:45.601730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:45.602602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 132/260] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:45.607741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.761886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f84f0-4dad-4c3f-80de-3b7d3a637760 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.801253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebc23a-73eb-414e-bd57-7cd494224ba2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.822900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.823519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.823789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.824050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.824714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:45.825068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.826378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.835587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 133/261] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:45.841153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.853537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.854831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.855335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.855901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.857250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.858950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.861117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 129/262] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:45.867602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.910278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:45.980551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:45.998513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fbd7-7881-4520-a36d-5c1b31d8c744 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.040176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96821-66d5-4180-a234-20668b6addd5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.063943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.064555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.064830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.065091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.078991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.079903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.080380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.080867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] PATH_INFO: `/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.082558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] RBAC: Authorizing `identity:get_user(user_id=1fecb9b00c7e4d6295f7e5603a4607ea)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.084327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.087113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 134/263] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/users/1fecb9b00c7e4d6295f7e5603a4607ea => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:46.096414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.216281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.217757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.243675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729995e5-b1fa-4571-9c98-7c04edc07d77 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.259340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.282341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.283451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.283931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.284393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.360750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.362570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.401869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.441635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 130/264] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:46.454741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.481499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 135/265] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:46.487371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.509208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.649193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7683f-16f1-491a-82f3-9e53c05c715f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.706339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bc3-b47b-405f-bb5f-0d972bc74e33 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.726364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.727094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.727368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.727637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.731280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.732716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.735777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 136/266] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:46.742458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.759224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.759850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.760122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.760387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.762567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.764034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.765656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 131/267] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:46.771453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.862979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.895094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.926903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee227c5-5ca5-4d9c-bddb-845a7a230b48 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.954359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d3d05-cc13-4774-942d-23f222b13f36 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.964931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.965614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.965960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.966297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.967109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=1fecb9b00c7e4d6295f7e5603a4607ea) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:46.970724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.970724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7f48f-3aec-4887-ab1d-fc024928245a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.977020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 137/268] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/role_assignments?user.id=1fecb9b00c7e4d6295f7e5603a4607ea&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:47.026793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.027678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.028099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.028532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.029416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=1fecb9b00c7e4d6295f7e5603a4607ea) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:47.029869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.031542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bb3db-3ca9-4d2c-9736-deec59f57672 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.043923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 132/269] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/role_assignments?user.id=1fecb9b00c7e4d6295f7e5603a4607ea&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:47.227381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a818-55dc-4b67-814e-a708addc8e05 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.227901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a818-55dc-4b67-814e-a708addc8e05 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.228054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a818-55dc-4b67-814e-a708addc8e05 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.229964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 138/270] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:47.235807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.236423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.236879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.263828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:47.265037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.337241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d985-5fb9-4295-9be2-5e2f37009e11 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.337852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d985-5fb9-4295-9be2-5e2f37009e11 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.338232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d985-5fb9-4295-9be2-5e2f37009e11 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.341226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 139/271] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:47.345841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.346058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 133/272] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:47.346158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.346506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.373711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:47.375006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.386103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.469630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 140/273] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:47.482358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.497129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.539903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0fb37-ce9b-48ca-bbf9-870e6002683f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.580738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.581458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.581805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.582145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.583768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.586203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9eb3f-9e45-4b56-815d-82beb5c6d5e8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.586894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.589071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 134/274] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:47.595315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070fab4-04ac-4081-af21-be1fa9af3179 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.595685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070fab4-04ac-4081-af21-be1fa9af3179 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.596063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070fab4-04ac-4081-af21-be1fa9af3179 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.597751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 135/275] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:47.598752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070fab4-04ac-4081-af21-be1fa9af3179 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.614899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b9113-84c4-4100-9040-8f1440b05639 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.657248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.657879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.658159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.658431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.669318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8dd82d66-25fe-4aef-b765-47e17771514d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:47.674014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:47.674947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:47.675810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 141/276] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:47.676453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.676904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.676904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.695501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:47.696678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.718682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070fab4-04ac-4081-af21-be1fa9af3179 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.736860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.755513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.756116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.756423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.756720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.757480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.759011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.769118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 142/277] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:47.778925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.842908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 136/278] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:47.862616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.885609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624502a8-0b2c-4ba8-9d74-f18b3162b0d0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.928936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.929972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.930248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.930515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.931220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:47.931517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.932937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d951e7e-ad93-4488-9028-8ca1a7560e15 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.947241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 143/279] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:47.947241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a664-c7d1-48e6-acd0-8c75a86ab9e0 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.947241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a664-c7d1-48e6-acd0-8c75a86ab9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.947241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a664-c7d1-48e6-acd0-8c75a86ab9e0 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.947241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 144/280] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:47.950518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a664-c7d1-48e6-acd0-8c75a86ab9e0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.991475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f101c61-9684-4967-a12b-58bed1154681 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.035552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.036813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.037254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.037559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.050756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-39b16d99-87dc-4358-844a-ce2c72ab360b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:48.056394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:48.057283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 137/281] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:48.059036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.059925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.059925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.073253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a664-c7d1-48e6-acd0-8c75a86ab9e0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.078242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:48.079461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.108963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.111681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.121071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.121710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.121997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.122824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] PATH_INFO: `/v3/users/7de4dba475944b34988896973ff84adb` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.139295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] RBAC: Authorizing `identity:get_user(user_id=7de4dba475944b34988896973ff84adb)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.140987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.142931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 145/282] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/users/7de4dba475944b34988896973ff84adb => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:48.143863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.163938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 138/283] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:48.164711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.270617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b1fd4-aa81-4d14-878b-93d02e81bbda None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.305442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf0d04-e972-4980-a88a-8a2e6aecd5e5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.309858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.310467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.310766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.311099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.311767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:48.312066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.313548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.323276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 146/284] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:48.329077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.347226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.347848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.348131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.348664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.349880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, user.id=7de4dba475944b34988896973ff84adb, scope.system=all) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:48.350834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.352834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.360252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 139/285] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/role_assignments?user.id=7de4dba475944b34988896973ff84adb&scope.system=all&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:48.361508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.387546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.400392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.449751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5702-2dbf-481f-b5e8-ec10808bbb43 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.487999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.488634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.488912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.489182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.497740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749537b9-fc82-4777-b5c4-c8b49e04335e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.501763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:48.508070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:48.509000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:48.509787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 147/286] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:48.515627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.539199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.540033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.540431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.540864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] PATH_INFO: `/v3/users/49d0820b3d0f48efbf18820a3671996b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.542204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] RBAC: Authorizing `identity:get_user(user_id=49d0820b3d0f48efbf18820a3671996b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.544085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.546035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 140/287] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/users/49d0820b3d0f48efbf18820a3671996b => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:48.553947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.653084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240d667-cd2a-4fd5-b80e-dc43730b3b22 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.664773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.690986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.691586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.691856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.692123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.692768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:48.693064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.694618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.709822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 148/288] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:48.712779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.715884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.721130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d83ae-4bb4-45e9-a26e-9d5c41d538d8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.767454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.768354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.768773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.769208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.914652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.921125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6056-79ba-4a64-923c-5c77a286201b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.921863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.031423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.031638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.032064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.032335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.032628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] PATH_INFO: `/v3/users/de8adf41f94e44098aa4ad45bf6a196f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.034202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] RBAC: Authorizing `identity:get_user(user_id=de8adf41f94e44098aa4ad45bf6a196f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.038144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.038144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 149/289] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/users/de8adf41f94e44098aa4ad45bf6a196f => generated 297 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:49.043604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.050004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 141/290] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:49.056606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.104448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.193330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9de67-0669-4758-8722-4ef8bb51ee5e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.231792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3804bd-cb85-4069-b881-48d19844a2c4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.232883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.233300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.233707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.235464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.236959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.238695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 150/291] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:49 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:49.247827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.293864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.294857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.295320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.295845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.298866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.302801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.307575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 142/292] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:49 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:49.313169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.396502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.452794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810500d-49e3-4536-952a-a4874f58038a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.475786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.501239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.502528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.503127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.503543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/de8adf41f94e44098aa4ad45bf6a196f/roles/e4d598d28bdb4541aebb9197eb7e37f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.505715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, user_id=de8adf41f94e44098aa4ad45bf6a196f, role_id=e4d598d28bdb4541aebb9197eb7e37f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.507087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fe7a3-c4b3-44fc-b621-cbe30c089433 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.508322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2358e7-ea31-41b3-9b85-fab2463d9944 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.550466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.551326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.551915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.552395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.553514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=49d0820b3d0f48efbf18820a3671996b) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:49.553968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.555605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fbe22-6f43-4976-a352-fd2e512fb093 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.562438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 143/293] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:49 2022] GET /identity/v3/role_assignments?user.id=49d0820b3d0f48efbf18820a3671996b&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:49.595055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 151/294] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:56:49 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/users/de8adf41f94e44098aa4ad45bf6a196f/roles/e4d598d28bdb4541aebb9197eb7e37f6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:50.246352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd371-8a02-442f-9b04-f9b35667f9ee None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.246896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd371-8a02-442f-9b04-f9b35667f9ee None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.247213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd371-8a02-442f-9b04-f9b35667f9ee None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.248797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 144/295] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:50.253929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bdb44-ddd3-4203-b4f3-b41182bac77b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.254223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bdb44-ddd3-4203-b4f3-b41182bac77b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.254503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bdb44-ddd3-4203-b4f3-b41182bac77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.273626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c71bdb44-ddd3-4203-b4f3-b41182bac77b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.274866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c71bdb44-ddd3-4203-b4f3-b41182bac77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.345138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 152/296] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:50.368526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd371-8a02-442f-9b04-f9b35667f9ee None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.482470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd371-8a02-442f-9b04-f9b35667f9ee None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.509789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.511096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b09b-a405-4143-a051-10c7ee14f739 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.511946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b09b-a405-4143-a051-10c7ee14f739 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.512418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b09b-a405-4143-a051-10c7ee14f739 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.514197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 153/297] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:50.518631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.519315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.519590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.519857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.520224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.520661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.521085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.532364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:50.538817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:50.539718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:50.540519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 145/298] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:50.546109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.546109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.546817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.623043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 154/299] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:50.642549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.668354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2845f-87d8-48c4-a518-044aae7b6ec6 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.672283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.712969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.713577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.713853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.714121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.714817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:50.715118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.717169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.733061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 146/300] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:50.734953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.762261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48723e9-85e4-4ad5-ba58-bb984a0701d3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.787012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.807195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.807820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.808098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.808366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.809456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.810974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32033e-0d49-4b27-8c8d-71c37f92654a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.812742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 155/301] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:50.813622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb933-1a95-4e14-b621-a9e75cc20c33 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.813895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb933-1a95-4e14-b621-a9e75cc20c33 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.814155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb933-1a95-4e14-b621-a9e75cc20c33 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.815349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 156/302] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:50.819694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb933-1a95-4e14-b621-a9e75cc20c33 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.851162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ce86-0d8d-43f4-a315-35c4c618bc08 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.887896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.888474 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.888754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.889031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] PATH_INFO: `/v3/users/394359e4bc114505a043c5883ab5b39a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.903252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=394359e4bc114505a043c5883ab5b39a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.904711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf83b3e-24b6-4881-801a-7f18ea7f0ea8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.906346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 147/303] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/users/394359e4bc114505a043c5883ab5b39a => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:50.906346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bf906-61da-4862-affa-305adabfd222 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.906346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bf906-61da-4862-affa-305adabfd222 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.906346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bf906-61da-4862-affa-305adabfd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.926206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-817bf906-61da-4862-affa-305adabfd222 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.927642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-817bf906-61da-4862-affa-305adabfd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.939564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.948387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb933-1a95-4e14-b621-a9e75cc20c33 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.986071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.986677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.986981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.987250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] PATH_INFO: `/v3/groups` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.987867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.989296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.997604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 148/304] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:50.999588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bf906-61da-4862-affa-305adabfd222 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.034187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.098799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 157/305] 10.222.0.36 () {60 vars in 1134 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/groups => generated 229 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:51.098799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.108204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bf906-61da-4862-affa-305adabfd222 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.151989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.152699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.153050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.153385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.154246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, user.id=394359e4bc114505a043c5883ab5b39a, scope.system=all) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:51.154676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.156494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.163590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 149/306] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/role_assignments?user.id=394359e4bc114505a043c5883ab5b39a&scope.system=all&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:51.236123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05826f-869f-4105-a4f0-287bab7c621a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.281162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.282127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.282613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.283049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.296817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:51.302504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:51.304304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:51.305083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:51.305674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 158/307] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:51.312209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.422055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.424970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff474-df3d-45aa-9ad7-608e13d130f2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.465877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.465877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.465877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.466215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.466660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:51.467037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.468415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.477902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 150/308] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:51.483496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.500382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.589098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64eb3b-f682-43be-9332-b8489ac3002c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.624006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.624767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.625880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.626150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] PATH_INFO: `/v3/users/7de4dba475944b34988896973ff84adb` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.627320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] RBAC: Authorizing `identity:get_user(user_id=7de4dba475944b34988896973ff84adb)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.628766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf1169-dec2-419b-9c62-55b659e2826b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.630350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 159/309] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/users/7de4dba475944b34988896973ff84adb => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:51.636463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.744232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835984-f7c6-4aa5-8812-78e05b2f7e63 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.780252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.780895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.781169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.781435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] PATH_INFO: `/v3/system/users/7de4dba475944b34988896973ff84adb/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.782931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7de4dba475944b34988896973ff84adb, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.784390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.785824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513671-ac9d-4027-b6a4-4782c64ae24b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.786107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513671-ac9d-4027-b6a4-4782c64ae24b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.786378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513671-ac9d-4027-b6a4-4782c64ae24b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.788659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 160/310] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:51.793578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.793856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.794123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.812524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:51.813734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.858137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 151/311] 10.222.0.36 () {58 vars in 1334 bytes} [Tue Aug 9 17:56:51 2022] PUT /identity/v3/system/users/7de4dba475944b34988896973ff84adb/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:51.889874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 161/312] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:51.915708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.947610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:52.035192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339134e-04af-41b6-b7c0-b2c425c40951 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:52.071495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:52.072352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.072771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.073194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.083944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:52.089326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:52.090394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:52.091401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:52.093595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 152/313] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:52.099172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:52.113840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:52.200727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3de972-c4b0-4fb6-ba71-0b105ef92e9a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:52.237208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:52.237852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.238082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.238349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.239037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:52.239332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:52.241489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:52.250155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 162/314] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:52.255326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:52.347536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:52.369078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e234d1-39ee-4d15-9621-37ed2c4161d3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:52.404322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:52.405192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.405614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.406068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] PATH_INFO: `/v3/users/de8adf41f94e44098aa4ad45bf6a196f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.407377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] RBAC: Authorizing `identity:get_user(user_id=de8adf41f94e44098aa4ad45bf6a196f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:52.408968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:52.411153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 153/315] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:52 2022] GET /identity/v3/users/de8adf41f94e44098aa4ad45bf6a196f => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:52.417745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:52.496716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:52.522315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876809ee-093d-46e1-b613-cbb0156cbfd7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:52.557098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:52.557744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.558018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.558286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.633134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:52.634526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:52.709996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:52.736764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 163/316] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:52.743172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:52.849774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775ca9-2c14-45f2-aeea-94a4731adbce None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:52.886672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:52.887244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.887513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.887775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.888864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:52.890304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d7c61-b48a-4281-aea8-8b8fff88d972 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:52.892233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 154/317] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:52 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:52.899220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.008585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879275-e24e-4893-abb4-f5b14f1fec56 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.044978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.045582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.045856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.046121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.046875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, scope.project.id=c340cf81fe874bdba01f7810de7bea28, user.id=de8adf41f94e44098aa4ad45bf6a196f) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:53.047178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.048607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2089589-ff83-4778-86d3-0061ee96e2a0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.055383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 164/318] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:56:52 2022] GET /identity/v3/role_assignments?user.id=de8adf41f94e44098aa4ad45bf6a196f&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:53.300813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b819a63-4e0f-426a-880f-dc8b565ac5ea None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.301200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b819a63-4e0f-426a-880f-dc8b565ac5ea None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.301373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b819a63-4e0f-426a-880f-dc8b565ac5ea None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.302680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 155/319] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:53.309368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.309652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.309863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39447-3ac6-4063-a2c7-6e846d662821 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.310042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.310190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39447-3ac6-4063-a2c7-6e846d662821 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.310400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39447-3ac6-4063-a2c7-6e846d662821 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.311848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 156/320] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:53.316470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.316744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.317011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.330216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:53.331937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.336545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:53.337672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.354931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.373134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.409688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 157/321] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:53.414673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 165/322] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:53.432740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.436934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.538158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0e71c-42b0-46b0-9ac4-c11f2a28bb22 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.550690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21e894-2ad1-4b15-8562-a79e0668d8f8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.576984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.577682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.577956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.578225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.579391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.581107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.585138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.585758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.586026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.586291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.588775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 158/323] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:53.590013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.596544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:53.602051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:53.603324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:53.604110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 166/324] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:56:53.608860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.618113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.641070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.703817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6e36-4368-4286-8a24-3fa7f2d4d235 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.714052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e853fe-9e39-4c46-afb1-d1bc8527237d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.741330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.742037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.742309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.742685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] PATH_INFO: `/v3/groups` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.743322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.744864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.750199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.750841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.751467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.751467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.752034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:53.752333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.753702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.762393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 167/325] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:53.768055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.844039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 159/326] 10.222.0.36 () {60 vars in 1134 bytes} [Tue Aug 9 17:56:53 2022] POST /identity/v3/groups => generated 226 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:53.874018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.884378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a249c3-b86a-471d-b97b-957d3dfa76d6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.922865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.923510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.924219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.924872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] PATH_INFO: `/v3/users/394359e4bc114505a043c5883ab5b39a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.926651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] RBAC: Authorizing `identity:get_user(user_id=394359e4bc114505a043c5883ab5b39a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.928313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be52a3f-7b4b-4333-9b31-bda7999497f9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.930336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 168/327] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/users/394359e4bc114505a043c5883ab5b39a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:56:53.936118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.025911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.051133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38307d-1237-4de7-812b-c7845c8f7ff8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.088196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.089356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.089634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.089905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] PATH_INFO: `/v3/system/users/394359e4bc114505a043c5883ab5b39a/roles/e4d598d28bdb4541aebb9197eb7e37f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.091348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=394359e4bc114505a043c5883ab5b39a, role_id=e4d598d28bdb4541aebb9197eb7e37f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.092731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.099057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323022ea-c4ee-4575-886d-2b59c0737944 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.099253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323022ea-c4ee-4575-886d-2b59c0737944 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.100043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323022ea-c4ee-4575-886d-2b59c0737944 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.101421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 169/328] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:54.106431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.106726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.107012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.125826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:54.127050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.153447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 160/329] 10.222.0.36 () {58 vars in 1334 bytes} [Tue Aug 9 17:56:53 2022] PUT /identity/v3/system/users/394359e4bc114505a043c5883ab5b39a/roles/e4d598d28bdb4541aebb9197eb7e37f6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:54.205155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 170/330] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:54.230840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.352656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197c80-3905-4878-9bba-209e94073528 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.387883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.388501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.388781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.389051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.400755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:54.406055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:54.406984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:54.407777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:54.409538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 161/331] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:54.414223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.518247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.531598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86890-6983-4440-8c0a-002e522abcb8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.567295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.567911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.568276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.569526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.570531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:54.571215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.574880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.586461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 171/332] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:54.587884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.600231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.707417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02360428-1556-4aa9-a869-a0b897f77fb6 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.750069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.750731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.751069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.751342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] PATH_INFO: `/v3/users/7de4dba475944b34988896973ff84adb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.752570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] RBAC: Authorizing `identity:get_user(user_id=7de4dba475944b34988896973ff84adb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.754033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a411-1cc0-4ea0-b09f-ff3ebb7f624f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.755937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 162/333] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:54 2022] GET /identity/v3/users/7de4dba475944b34988896973ff84adb => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:54.762128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.905988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4fe5-f1d3-4227-8620-bd22fc4e8a6d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.942428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.943063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.943869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.944147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.944876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] RBAC: Adding query filter params (role.id=8e6f9bbf4e854915a68a5b685ccb788d, user.id=7de4dba475944b34988896973ff84adb, scope.system=all) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:54.945179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.946641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5a3c4-5635-42b7-a3b4-d7c6c6169fc6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.966977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 172/334] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Aug 9 17:56:54 2022] GET /identity/v3/role_assignments?user.id=7de4dba475944b34988896973ff84adb&scope.system=all&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:56.635519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e7ec-136d-4009-b88d-195fabfa0076 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.636058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e7ec-136d-4009-b88d-195fabfa0076 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.636058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e7ec-136d-4009-b88d-195fabfa0076 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.637513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 163/335] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:56.651685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ad76-e3fc-469d-9834-d09ab754041d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.651685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ad76-e3fc-469d-9834-d09ab754041d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.651685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ad76-e3fc-469d-9834-d09ab754041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.679877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-aee8ad76-e3fc-469d-9834-d09ab754041d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:56.681073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aee8ad76-e3fc-469d-9834-d09ab754041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:56.683155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:56.766603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 173/336] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:56.804610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e7ec-136d-4009-b88d-195fabfa0076 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.811749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c2f6-0240-47c6-a69a-69339f3458a8 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.812036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c2f6-0240-47c6-a69a-69339f3458a8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.812312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c2f6-0240-47c6-a69a-69339f3458a8 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.814511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 174/337] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:56.819706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713c57-6eb8-4c16-b30f-7ac61d835ab0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.819988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713c57-6eb8-4c16-b30f-7ac61d835ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.820263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713c57-6eb8-4c16-b30f-7ac61d835ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.838991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a3713c57-6eb8-4c16-b30f-7ac61d835ab0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:56.840707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a3713c57-6eb8-4c16-b30f-7ac61d835ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:56.917789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 175/338] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:56:56.921009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee8a9-9f34-4994-9e9b-fed218746e68 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.921704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee8a9-9f34-4994-9e9b-fed218746e68 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.923497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee8a9-9f34-4994-9e9b-fed218746e68 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.925886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 176/339] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:56.926471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:56.935003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.935565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.935998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.968158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:56.970680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.027519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e7ec-136d-4009-b88d-195fabfa0076 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.049130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.049895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 177/340] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:57.059340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.159955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.160595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.160865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.161127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.169297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83084ffd-904d-4148-ac19-49f2c0b92984 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.180042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:57.185150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:57.186010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:57.187208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 164/341] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:57.195185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.209930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.211004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.211700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.212582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.227518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:57.233442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:57.234428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:57.235252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:57.237643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 178/342] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:57.238940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.339741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.356458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051bdcd-ef7b-43ef-8bdc-4c0791b7a5d0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.364825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15aa42-94b8-4d3a-85ab-0a2ebe389b0c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.392280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.398760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.399414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.399678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.399939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.412863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.413283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:57.414074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.414558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.415049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.415981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:57.416482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.419085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccb094-c3fd-40ea-a8ba-c3e94f2d1b70 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:57.422673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:57.423587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 165/343] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:57.424467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.433849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 179/344] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:57.435599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c21e7a-f9f4-4a26-abc8-adf6476636c2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.436081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c21e7a-f9f4-4a26-abc8-adf6476636c2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.436545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c21e7a-f9f4-4a26-abc8-adf6476636c2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.438228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 180/345] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:57.439398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c21e7a-f9f4-4a26-abc8-adf6476636c2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.559210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e98ecf-d0c5-44b5-a4f8-575722b43e4e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.563494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c21e7a-f9f4-4a26-abc8-adf6476636c2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.587245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.607383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.608258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.608965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.609359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.609526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.610160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:57.610829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.611099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.611390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.612110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:57.612401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.613824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.615857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.617887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e4692-d914-4225-8db9-57472c32cde9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.624715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 166/346] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:57.627235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.629564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 181/347] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:57.631354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.631966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.632438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.655404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:57.657656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.695464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.739472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 182/348] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:57.741264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.746781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1616-92c2-4eaa-be67-067690855581 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.795567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.796243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.796518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.796809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.865348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50cdc8-5779-462d-9376-e518c1465dfa None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.874850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.876402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.882345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.909908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.911041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.911697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.912469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.942958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.994698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.996590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.998744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 167/349] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:57.999557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.107604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7244002-164d-4e37-85ad-6fdb8bd2bd45 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.114746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 183/350] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:58.114746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.131960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.152147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.153150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.153631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.154101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] PATH_INFO: `/v3/users/394359e4bc114505a043c5883ab5b39a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.155609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] RBAC: Authorizing `identity:get_user(user_id=394359e4bc114505a043c5883ab5b39a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.157261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.159534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 168/351] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/users/394359e4bc114505a043c5883ab5b39a => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.160582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.188096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.257194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681f06b-5df2-441b-8bea-50f2bf3ba309 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.279579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246521f-f8cd-45cd-8bee-dcfa33b24da1 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.306017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.306654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.307023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.307298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.320982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:58.325061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-d672d3e1-ee95-456f-a70b-74ce357b6f50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:58.325772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:58.326658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:56:58.327469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:56:58.328060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.328060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.328060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.328237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 184/352] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:58.328883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c5ba-ade7-46c5-8777-766052ec606c None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.328883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c5ba-ade7-46c5-8777-766052ec606c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.329047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c5ba-ade7-46c5-8777-766052ec606c None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.329208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.330355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 185/353] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:58.330806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.331299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c5ba-ade7-46c5-8777-766052ec606c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.336461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 169/354] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.337117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.435837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.463742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.477169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0adb0f-7d8c-4217-93aa-368c7ab99323 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.480862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c5ba-ade7-46c5-8777-766052ec606c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.523656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.524646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.525027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.525396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.526627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.527598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.528110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.528226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.528461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.528728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.529449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] RBAC: Adding query filter params (role.id=e4d598d28bdb4541aebb9197eb7e37f6, user.id=394359e4bc114505a043c5883ab5b39a, scope.system=all) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:58.529938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.530046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 186/355] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.531227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.534978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.548086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 170/356] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/role_assignments?user.id=394359e4bc114505a043c5883ab5b39a&scope.system=all&role.id=e4d598d28bdb4541aebb9197eb7e37f6 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.549421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.674275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284165c-9412-4550-9d85-6d687bad58df None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.681564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6103b7-6bc0-499a-bda8-a1e5b9587b59 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.716302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.717566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.719158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.719158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.719420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:58.719818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.721651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ed58a-7b58-44c9-b25a-11085670e698 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.722929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.724201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.725154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.725760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.732712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.733684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 187/357] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.734591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.735393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.735916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.736383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.736622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 171/358] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.737388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.745470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.756229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.760483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:58.761916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:58.851080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 188/359] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:58.851965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.867612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e211-5e55-4d22-83d6-aad17a685ac5 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.916812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.917724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.918182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.918549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.920230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.922049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.927028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 172/360] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:58.928100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.985213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d2d21-cd0a-4d2c-a317-33ce64b54e0a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.026366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.032301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.032527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.033155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.033365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.033636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.054208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85c134-f209-42af-9e49-0bdf0f99d800 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.091278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.091882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.092156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.092423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.093141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=bc75ae0177fe43469516093786d30add) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:59.093446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.095017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.102151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 173/361] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:59.102998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.114179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.116430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.251035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809e493-a181-4068-b081-02ed78e103b8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.255345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 189/362] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:56:59.256664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.298489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.302239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.303990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.304371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.304641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.321007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4c8ff47e-d28e-4906-bc2c-4d4547fc9a6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:56:59.326397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:56:59.327435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 174/363] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:56:59.328979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90351a-cd73-4643-96e3-6b88bdff9163 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.329703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90351a-cd73-4643-96e3-6b88bdff9163 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.329703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90351a-cd73-4643-96e3-6b88bdff9163 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.334769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 175/364] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:56:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:59.334769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90351a-cd73-4643-96e3-6b88bdff9163 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.337240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.392906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fb22-ef01-4037-8551-40d245e850c5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.434152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.434793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.435076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.435348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.437803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=bc75ae0177fe43469516093786d30add) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:59.437803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.437986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.453986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 190/365] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:59.455134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.489847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90351a-cd73-4643-96e3-6b88bdff9163 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.542266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.543509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.543895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.544280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.546274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.547922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cba60a-83be-4c7f-a605-c78747e83bc2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.550001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 176/366] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:59.551973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.552394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.553280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.578791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:59.580220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:59.602878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.610954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd743-09cc-4eb4-99e7-32ed2a20a258 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.667878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.668573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.668861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.669198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.669877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:59.670181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.672601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.684462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 191/367] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:59.685600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.695405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 177/368] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:56:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:56:59.696683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.699531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.813646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4879-ca87-49d5-a403-3b132c08a8cc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.828292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153efa1-8197-48cf-b25c-508b2295edb5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:59.855221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.856195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.856689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.857150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.873967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:59.874607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:59.874921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:59.875220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:59.877125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.878883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.881991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 192/369] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:59.882879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:59.920091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:59.942830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.945012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.948241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.017348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28bf0e-4cfb-4afe-83fd-1922d856230a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.067685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 178/370] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:00.070123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.076420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.077065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.077345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.077620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.092062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:00.098543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:00.099606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 193/371] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:00.101023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.199277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867be3-1fef-4ef9-a3ee-bff08ef7765e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.201315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.224728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.243268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1c55-8b2f-4b53-8965-14201b370eb1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.243726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.244421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.244948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.245231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.245963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=c340cf81fe874bdba01f7810de7bea28) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:00.246271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.247835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.254785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 179/372] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:00.256183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.288356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.289461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.289941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.290464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.292838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.294606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.301289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 194/373] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:00.303420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.389451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3c0e-bed5-4bc7-bc0f-0e38e9ba4388 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.433960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.434993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.435472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.435994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.436895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:00.437404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.439120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.444302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf052-3ef9-4178-b7d3-7500997b8e5b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.450767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 180/374] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:00.459856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.492658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.493431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.493835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.494133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.495547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.497333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.500418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.504910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 195/375] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:00.514242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:00.553421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.622096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e6407-b489-490f-b974-0d71cbd9f4eb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.670383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6c9b5-47cb-4a84-850d-ed1c90204439 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:00.672628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.674437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.675006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.675628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.723223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:00.725493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:00.726046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:00.726511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:00.727935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=c340cf81fe874bdba01f7810de7bea28) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:00.728455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.730141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.740415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 196/376] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:00.829139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:00.830614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:00.873767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:00.971211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 181/377] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:00.978529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:01.107365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:01.119391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d97c9-fd1b-4de0-aff6-990043262431 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:01.160565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:01.161206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.161482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.161795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] PATH_INFO: `/v3/projects/c024e80f00ae4274ad841a20660b3ad2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.163354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=c024e80f00ae4274ad841a20660b3ad2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:01.164806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:01.166789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 197/378] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:00 2022] GET /identity/v3/projects/c024e80f00ae4274ad841a20660b3ad2 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:01.172371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:01.301623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c57d-1998-48c6-9805-53d96ad04f6a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:01.336777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:01.345787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:01.346612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.347059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.347495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] PATH_INFO: `/v3/groups/fdf0d1614746400eb9ed450a111bb6de` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.354529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] RBAC: Authorizing `identity:get_group(group_id=fdf0d1614746400eb9ed450a111bb6de)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:01.356225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad38a4-ca5f-4ee2-b6d8-edfda5dfedb1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:01.358246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 182/379] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:01 2022] GET /identity/v3/groups/fdf0d1614746400eb9ed450a111bb6de => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:01.367799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:01.490377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51933c-ba83-491b-bedb-2a0493d7d5a8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:01.520310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ddee-9b35-44f2-a79b-6266dcfd5a3b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.520616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ddee-9b35-44f2-a79b-6266dcfd5a3b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.520854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ddee-9b35-44f2-a79b-6266dcfd5a3b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.523144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 183/380] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:01.528576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.528849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.529119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.554696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:01.556838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.557176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.557453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.558189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] RBAC: Adding query filter params (group.id=fdf0d1614746400eb9ed450a111bb6de, role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=c024e80f00ae4274ad841a20660b3ad2) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:01.558496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:01.559972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:01.565768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:01.567289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:01.578835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 198/381] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:01 2022] GET /identity/v3/role_assignments?group.id=fdf0d1614746400eb9ed450a111bb6de&scope.project.id=c024e80f00ae4274ad841a20660b3ad2&role.id=7409e93d755b445b891c0195b28879f6 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:01.668622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 184/382] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 17:57:01.692751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:01.717198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:01.829161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8210fc0-66e9-42fb-a496-be8d1670fca4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:01.874344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:01.875017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:01.875291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:01.875559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:01.895659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:01.901769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:01.903926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:01.905830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 199/383] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:01.910736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.039605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b81073-35ff-48fb-80a8-ad0fbd8a4059 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.064254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:02.078143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.078867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.079153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.079472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.080148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:02.080447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:02.081869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a66552-4422-4f70-89d8-8dba603794a3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:02.094844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 185/384] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:57:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:02.097079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.126388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778927-a067-430d-bed6-b7814e8d83c9 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.126687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778927-a067-430d-bed6-b7814e8d83c9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.126992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778927-a067-430d-bed6-b7814e8d83c9 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.128518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 186/385] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:02.134368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.134644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.134970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.163584 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:02.168891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:02.244473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:02.281180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 187/386] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 17:57:02.285868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed779c-4ea6-46a4-b4fe-62c060407ed1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.325030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.330108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.330684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.331053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.331325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.332420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:02.334606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:02.342744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 200/387] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:02 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:02.344189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.467295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1dd8-d78e-4ee7-a49c-5cbcf1d99253 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.483922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:02.503770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca623-cc48-46a9-af58-853f82c51cb6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.504751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.506076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.506655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.507146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.517891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:02.524056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:02.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:02.526133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 188/388] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:02.539719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.580846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.581412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.581688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.581955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.587463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:02.588915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:02.594728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 201/389] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:02 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:02.604782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.639853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:02.712157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886bd0-1103-41fb-abf9-0fc325faa7a3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.789023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.790026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.790497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.791007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.792798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:02.794529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:02.796154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:02.811147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 189/390] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:02.812932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763687f4-fe83-4e32-b1d5-6a6513009394 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.821034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:02.870551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:02.892450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:02.894364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:02.895665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:02.897064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/groups/3251e8c6069641a99a9912a6cd37d05c/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:02.903484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, group_id=3251e8c6069641a99a9912a6cd37d05c, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:02.904936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67ad37-a79f-4c30-b496-26474bd79a92 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:02.965760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53c16-f6fc-46b5-adea-571bb0fc3553 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:02.990127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 202/391] 10.222.0.36 () {58 vars in 1442 bytes} [Tue Aug 9 17:57:02 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/groups/3251e8c6069641a99a9912a6cd37d05c/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:03.013659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.014765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.015276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.015747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.017111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.018890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.022329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e3ac6-7530-4816-b719-eef2b181aba6 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.022645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e3ac6-7530-4816-b719-eef2b181aba6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.022865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e3ac6-7530-4816-b719-eef2b181aba6 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.023927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 190/392] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:02 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:03.024978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 203/393] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:03.029589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.030001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.030283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.030552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.063847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:03.071341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:03.208873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 204/394] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:03.216728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c461c17-ffa5-4598-99cc-b94c0488866b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.220017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.232162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.241447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.261667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.262453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.263089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.263780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.265247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.267178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.270216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 191/395] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:03.278612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.352422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fba96-aab9-44ee-abf9-cd64a14382c2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.391807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.392442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.392719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.392989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.412792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:03.417659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:03.418627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:03.419447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:03.421911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 205/396] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:03.426544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.460215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308cbe-def5-4182-a0c8-b80d791ebfee None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.502983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.503230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.503491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.505777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add/groups/3251e8c6069641a99a9912a6cd37d05c/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.510061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc75ae0177fe43469516093786d30add, group_id=3251e8c6069641a99a9912a6cd37d05c, role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.513825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1e96a-41dd-4a2f-a358-6585e07b6cd0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.530901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.568991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3923d-aef4-47b7-8af3-8c758596db8c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.604095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 192/397] 10.222.0.36 () {58 vars in 1442 bytes} [Tue Aug 9 17:57:03 2022] PUT /identity/v3/projects/bc75ae0177fe43469516093786d30add/groups/3251e8c6069641a99a9912a6cd37d05c/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:03.608159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.609113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.609719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.610258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.611216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:03.611336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa42eb-1ae7-41ec-8188-984e962f1370 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.611612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa42eb-1ae7-41ec-8188-984e962f1370 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.611954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa42eb-1ae7-41ec-8188-984e962f1370 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.612302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.613734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 193/398] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:03.614301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.619995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.627276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.627276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.627475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 206/399] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:03.632481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.668318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:03.669544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:03.733573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.794923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6104-360d-4474-a218-f5d7b0b5c1f3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.816303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 194/400] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:03.827572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.838834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.839587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.839987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.840432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.842724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.842858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.844304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.846072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 207/401] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:03.857935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.967899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22fe5-79cd-4052-81ef-18e00c2906c5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.992958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadb9e8-2d02-436d-be44-a53448d0b526 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.015370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.016108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.016380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.016645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.028503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:04.033377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:04.034249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:04.035070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 195/402] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:04.041187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.043644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.044787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.046222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.046694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.048451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.051348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.054857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 208/403] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:04.064449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.078736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.156882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953f53-de2b-4025-ae54-12466ab75ebb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.166230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.195203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.195846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.196090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.196364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.197032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:04.197335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.198730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.222754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 196/404] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:04.231560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.246383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df20161-def0-42a8-85a1-9e1905ac511f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.313601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.314264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.314540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.314834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/groups/3251e8c6069641a99a9912a6cd37d05c/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.323484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, group_id=3251e8c6069641a99a9912a6cd37d05c, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.324939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2452efe-33e8-49dd-a805-13b38189bfae None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.347095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.357739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a06a84-1ffe-4a40-92f3-d244adb910dc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.391674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.392295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.392570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.392837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.393944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.395419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89b98c-0c8e-4caf-b090-b9bc792d90b9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.397700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 197/405] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:04.398824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985877f-1630-471f-8d9e-9dd5c0c05c66 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.399105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985877f-1630-471f-8d9e-9dd5c0c05c66 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.399374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985877f-1630-471f-8d9e-9dd5c0c05c66 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.400550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 198/406] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:04.405237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985877f-1630-471f-8d9e-9dd5c0c05c66 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.433114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 209/407] 10.222.0.36 () {58 vars in 1442 bytes} [Tue Aug 9 17:57:04 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/groups/3251e8c6069641a99a9912a6cd37d05c/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:04.434344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.434627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.434924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.454304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:04.455544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:04.595883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 210/408] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:04.602277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985877f-1630-471f-8d9e-9dd5c0c05c66 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.651208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.651847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.652176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.652451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.653766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.655335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.658283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 199/409] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:04.664687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.664819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.689335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.724644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.775867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fce58-b82c-4567-9669-3bce6c1ac9b3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.776092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e38700-9ea6-4cbf-ab80-f45bd309c13e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.812173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.812801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.812926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.813123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.813363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.813602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.814059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.814351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28/groups/3251e8c6069641a99a9912a6cd37d05c/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.816550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c340cf81fe874bdba01f7810de7bea28, group_id=3251e8c6069641a99a9912a6cd37d05c, role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.818308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.825276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:04.830510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:04.831408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:04.832184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 200/410] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:04.837774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.922024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 211/411] 10.222.0.36 () {58 vars in 1442 bytes} [Tue Aug 9 17:57:04 2022] PUT /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28/groups/3251e8c6069641a99a9912a6cd37d05c/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:04.944414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:04.969060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473f470-5048-4753-8f1b-6384eee0c9e6 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:05.018113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:05.019051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.019499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.019919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.021298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:05.021760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:05.023502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:05.040060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 201/412] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:05.046035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:05.174752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12866ba-2181-4edb-9fdb-b255aa31cdc5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:05.188359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:05.211418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:05.214589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.214589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.214589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] PATH_INFO: `/v3/groups/fdf0d1614746400eb9ed450a111bb6de` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.214589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] RBAC: Authorizing `identity:get_group(group_id=fdf0d1614746400eb9ed450a111bb6de)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:05.215883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:05.217763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 212/413] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:05 2022] GET /identity/v3/groups/fdf0d1614746400eb9ed450a111bb6de => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:05.223073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:05.362119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adebb45-4c77-4c2d-ba97-a8552894537a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:05.398447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:05.399496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.399997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.400462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] PATH_INFO: `/v3/projects/c024e80f00ae4274ad841a20660b3ad2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.401789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] RBAC: Authorizing `identity:get_project(project_id=c024e80f00ae4274ad841a20660b3ad2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:05.403470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1c02a-3ff0-4fac-a0ed-28ef034fdee6 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:05.405666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 202/414] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:05 2022] GET /identity/v3/projects/c024e80f00ae4274ad841a20660b3ad2 => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:05.412404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:05.541012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1fe32-a67c-4c6e-aea5-9736c7637dc1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:05.582352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:05.583147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.584166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.584446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] PATH_INFO: `/v3/projects/c024e80f00ae4274ad841a20660b3ad2/groups/fdf0d1614746400eb9ed450a111bb6de/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.590750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c024e80f00ae4274ad841a20660b3ad2, group_id=fdf0d1614746400eb9ed450a111bb6de, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:05.592223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:05.627040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392651-3a56-420c-9bbe-dc0c5f00f7e5 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.627882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392651-3a56-420c-9bbe-dc0c5f00f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.628385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392651-3a56-420c-9bbe-dc0c5f00f7e5 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.630036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 203/415] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:05.637324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.643664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.643664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.659629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:05.661816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:05.666663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:05.669728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 213/416] 10.222.0.36 () {58 vars in 1442 bytes} [Tue Aug 9 17:57:05 2022] PUT /identity/v3/projects/c024e80f00ae4274ad841a20660b3ad2/groups/fdf0d1614746400eb9ed450a111bb6de/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:05.750617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 204/417] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:05.772092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:05.819401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:05.885376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5c74e-59d6-4caa-b399-0be68c9576ce None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:05.929682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:05.930309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:05.930581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:05.930875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:05.947753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-461f01ad-d48b-4f3a-a2b4-5f2695436e18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:05.952403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:05.953467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:05.954418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 214/418] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:05.961807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.037624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1595a3-a8e5-48bb-989d-057771ba9295 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.038010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1595a3-a8e5-48bb-989d-057771ba9295 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.038285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1595a3-a8e5-48bb-989d-057771ba9295 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.044184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 215/419] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:06.049362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150acf7c-7dad-430a-b170-986413a0efec None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.050308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150acf7c-7dad-430a-b170-986413a0efec None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.050827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150acf7c-7dad-430a-b170-986413a0efec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.067149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936ff7f-afa5-4db2-8a02-60c1a1068d92 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.069494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-150acf7c-7dad-430a-b170-986413a0efec None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:06.070963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-150acf7c-7dad-430a-b170-986413a0efec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:06.103829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.104796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.105261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.105731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.106576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:06.107089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:06.108642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:06.117596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 205/420] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:57:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:06.132066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.142282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.148440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 216/421] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:06.248737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150acf7c-7dad-430a-b170-986413a0efec None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.269459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5aa63-7e08-48ef-852d-a736814764b8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.309069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.331061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.331716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.331999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.332269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.424388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:06.425174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150acf7c-7dad-430a-b170-986413a0efec None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.425898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:06.430448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.477936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.478577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.478886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.479160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.501458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:06.506397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:06.507305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:06.508169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 217/422] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:06.514550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.588925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 206/423] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:06.594498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.687071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.696432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e098-dbd5-491e-98b0-de06b348c289 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.732637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.733247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.733530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.733804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.734471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:06.734800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:06.736186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:06.744822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 218/424] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:06.751017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.791253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814e1e-4453-42ae-a0d1-4e4a3387f5cf None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.795038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.851510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.852064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.852344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.852615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.853774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:06.855259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:06.857164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 207/425] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:06.858362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a80cd-9a47-4c48-8f7a-2ceaba562ed3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:06.862465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:06.898297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:06.898949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:06.899232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:06.899503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:06.931134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:06.978645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:06.980360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df107c92-db8d-46fa-86cd-fcd5759f60ba None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.063242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5927a81-d40c-4d03-9f7f-4252267314a4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.104052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.104660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.104913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.105129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.106445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.107951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.110418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 208/426] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:07.122757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.171579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 219/427] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:07.177198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaaaff-5172-4da5-855d-6ca794b7aef2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.177198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaaaff-5172-4da5-855d-6ca794b7aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.177198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaaaff-5172-4da5-855d-6ca794b7aef2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.178618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 220/428] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:07.179517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaaaff-5172-4da5-855d-6ca794b7aef2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.205638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.248225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.272729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364d15-d6c2-414e-9cf7-fd3a58364acb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.312066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaaaff-5172-4da5-855d-6ca794b7aef2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.314246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.314955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.315325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.315667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.316458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=bc75ae0177fe43469516093786d30add) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:07.316836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.319010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f5249-5659-4ca3-89d3-56e9a9d9773f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.325559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 209/429] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:07.326964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.327319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.327660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.346958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:07.348483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:07.352302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.353090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.353364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.353629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] PATH_INFO: `/v3/projects/bc75ae0177fe43469516093786d30add` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.354920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] RBAC: Authorizing `identity:get_project(project_id=bc75ae0177fe43469516093786d30add)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.356360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.358427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 221/430] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/projects/bc75ae0177fe43469516093786d30add => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:07.363539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.423852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 210/431] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:07.460173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.482021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d5e7c-2f00-49ae-8006-bdc99871f7ba None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.507849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.517208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.532419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.533113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.533387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.533715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.538201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.539757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.541535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 222/432] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:07.548315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.640563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89e2a-8c76-47e3-b1e4-54b7d1304feb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.688308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.688668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.688946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.689217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.698245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913916ba-265d-4409-9b42-eb90d35eab18 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.703419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-ee062793-754a-400c-86d1-a2bace34c88e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:07.709542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:07.710482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:07.711431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 211/433] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:07.712102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798210-0045-40f8-b3ee-6e171d6826a3 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.712884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798210-0045-40f8-b3ee-6e171d6826a3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.713259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798210-0045-40f8-b3ee-6e171d6826a3 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.714642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 212/434] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:07.719605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5798210-0045-40f8-b3ee-6e171d6826a3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.753998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.754579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.754899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.755858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.756880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=bc75ae0177fe43469516093786d30add) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:07.757186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.758625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c4005-524d-4438-83d6-3690887d243a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.766863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 223/435] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=bc75ae0177fe43469516093786d30add&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:07.768833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.769402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.769890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.803044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:07.805181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:07.846345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.860151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5798210-0045-40f8-b3ee-6e171d6826a3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:07.876614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:07.910272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:07.910893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 224/436] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:07.912123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:07.912601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:07.913062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:07.913944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:07.914439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:07.916082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:07.932037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 213/437] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:07.936661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:07.939850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.071090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a66edc-8bec-465b-94ed-27d5e5fa3fc2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.092627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e78ee-4b79-41b5-a808-79cf4b85eb45 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.121082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.121717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.121988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.122261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.134038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4a5e6424-1f81-4ede-bdb4-c0997f5b71af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:08.138842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:08.139710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:08.140519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:08.141115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.141515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.141628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.141967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.142540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 225/438] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:08.143792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8562d-5ab0-45e0-a779-10dcb2bf02cd None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.144064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8562d-5ab0-45e0-a779-10dcb2bf02cd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.144327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8562d-5ab0-45e0-a779-10dcb2bf02cd None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.146291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 226/439] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:08.147310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8562d-5ab0-45e0-a779-10dcb2bf02cd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.177776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.203070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.231031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.232496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.273683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8562d-5ab0-45e0-a779-10dcb2bf02cd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.307058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.307657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.307934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.308206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.308853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:08.309152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.310519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9f358-41bb-4258-8775-02bb2dbbb33b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.318756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 227/440] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.320076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.320356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.320622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.338621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:08.339829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:08.342465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 214/441] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:08.343419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.399446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.408252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 228/442] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:08.409080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.443426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.444341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a403c4-84dc-4eb1-a05e-320cc1489c88 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.478837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.479443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.479723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.479992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.511997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691ddc7-3164-4649-9cce-2c719a1f71e9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.545427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.546046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.546321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.546589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.547305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.547717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.548773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.549143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.556442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 229/443] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.556442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.602864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.647967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 215/444] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:08.648796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.653497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1c012-a142-43ed-8f44-60446a5320d1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.655061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.689186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605286c5-0076-4094-8c24-438846db5889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.689816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605286c5-0076-4094-8c24-438846db5889 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.690244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605286c5-0076-4094-8c24-438846db5889 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.690868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605286c5-0076-4094-8c24-438846db5889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.701200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605286c5-0076-4094-8c24-438846db5889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-605286c5-0076-4094-8c24-438846db5889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:08.705852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:08.706738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:08.707546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 230/445] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:08.708272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605286c5-0076-4094-8c24-438846db5889 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.751517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d4a3-189f-4efb-bbd2-6a617d9c97d1 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.784590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.785205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.785477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.785742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.787554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.789018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.790561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 216/446] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.791335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.808845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605286c5-0076-4094-8c24-438846db5889 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.813799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.843289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.843913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.844193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.844468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] PATH_INFO: `/v3/projects/c340cf81fe874bdba01f7810de7bea28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.845588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] RBAC: Authorizing `identity:get_project(project_id=c340cf81fe874bdba01f7810de7bea28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.847072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.853787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 231/447] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/projects/c340cf81fe874bdba01f7810de7bea28 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.853787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.882821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:08.887810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc1d0-9983-44e3-ac7d-7bb5f320b12d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.923453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.924051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.924326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.924595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.925250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:08.925547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.927485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.935328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 217/448] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.936107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:08.952891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b7c5-d2db-4ed2-be44-45a19c57de1b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:08.988003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:08.988003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:08.988003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:08.988003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:08.989005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=8e6f9bbf4e854915a68a5b685ccb788d, scope.project.id=c340cf81fe874bdba01f7810de7bea28) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:08.990017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:08.991605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:08.997219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 232/449] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=8e6f9bbf4e854915a68a5b685ccb788d => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:08.998012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:09.035295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:09.052344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea5d9d-2a84-4d89-95c8-525eb63037ac None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.090512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.091172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.091452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.091720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] PATH_INFO: `/v3/groups/3251e8c6069641a99a9912a6cd37d05c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.092896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] RBAC: Authorizing `identity:get_group(group_id=3251e8c6069641a99a9912a6cd37d05c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.094372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.096022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 218/450] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/groups/3251e8c6069641a99a9912a6cd37d05c => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:09.102497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:09.115899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:09.123185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f0ee-28b4-4aa0-af37-3c16730c707a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.164033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.165069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.165069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.165266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.225834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db797d-99a7-4d00-aea4-c5288414cf9a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.252348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.253815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.266532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.267166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.267446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.267716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.268440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] RBAC: Adding query filter params (group.id=3251e8c6069641a99a9912a6cd37d05c, role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=c340cf81fe874bdba01f7810de7bea28) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:09.268743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.270173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.277403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 219/451] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:09 2022] GET /identity/v3/role_assignments?group.id=3251e8c6069641a99a9912a6cd37d05c&scope.project.id=c340cf81fe874bdba01f7810de7bea28&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:09.364206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:09.373534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 233/452] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:09.379480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:09.400603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:09.485545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85ecd-d15c-435a-9076-13192b036576 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.519952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.521103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.521456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.521752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] PATH_INFO: `/v3/projects/c024e80f00ae4274ad841a20660b3ad2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.522915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] RBAC: Authorizing `identity:get_project(project_id=c024e80f00ae4274ad841a20660b3ad2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.524351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.525967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 220/453] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:57:09 2022] GET /identity/v3/projects/c024e80f00ae4274ad841a20660b3ad2 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:09.531633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:09.633396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf4c1-c24f-494a-b760-d22cdd6d3683 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.669217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.669828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.670103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.670370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] PATH_INFO: `/v3/groups/fdf0d1614746400eb9ed450a111bb6de` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.671480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] RBAC: Authorizing `identity:get_group(group_id=fdf0d1614746400eb9ed450a111bb6de)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.672911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac54bca-4bfb-4136-bc10-903ea1313089 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.674457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 234/454] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Aug 9 17:57:09 2022] GET /identity/v3/groups/fdf0d1614746400eb9ed450a111bb6de => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:09.680445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:09.761789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:09.777537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243633f2-0162-4697-a6dd-cbd40e1c93e2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:09.810042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:09.810621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:09.810919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:09.811185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:09.811888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] RBAC: Adding query filter params (group.id=fdf0d1614746400eb9ed450a111bb6de, role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=c024e80f00ae4274ad841a20660b3ad2) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:09.812184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:09.813599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61379fa9-3a9c-4d78-abab-5867abb2a0bf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:09.819303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 221/455] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Aug 9 17:57:09 2022] GET /identity/v3/role_assignments?group.id=fdf0d1614746400eb9ed450a111bb6de&scope.project.id=c024e80f00ae4274ad841a20660b3ad2&role.id=7409e93d755b445b891c0195b28879f6 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:12.212728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fb9c5-a33f-4061-9c12-2b3c5eb8529a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.213253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fb9c5-a33f-4061-9c12-2b3c5eb8529a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.213253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fb9c5-a33f-4061-9c12-2b3c5eb8529a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.214858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 235/456] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:12.220431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f85a2-dbfa-4929-8879-ecf7c04c9e24 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.220895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f85a2-dbfa-4929-8879-ecf7c04c9e24 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.220973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f85a2-dbfa-4929-8879-ecf7c04c9e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.251712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3f2f85a2-dbfa-4929-8879-ecf7c04c9e24 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:12.252892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3f2f85a2-dbfa-4929-8879-ecf7c04c9e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:12.349505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 222/457] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:12.373737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fb9c5-a33f-4061-9c12-2b3c5eb8529a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:12.402880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b39f8-ab05-4efc-be1c-f5e3db5b48e0 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.403171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b39f8-ab05-4efc-be1c-f5e3db5b48e0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.403429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b39f8-ab05-4efc-be1c-f5e3db5b48e0 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.404983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 223/458] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:12.410127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcabc11-6280-4beb-b512-a3904444499f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.410405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcabc11-6280-4beb-b512-a3904444499f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.410672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcabc11-6280-4beb-b512-a3904444499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.438856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cbcabc11-6280-4beb-b512-a3904444499f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:12.440362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cbcabc11-6280-4beb-b512-a3904444499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:12.467141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:12.497097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fb9c5-a33f-4061-9c12-2b3c5eb8529a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:12.532478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:12.533101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.533373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.533640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.544732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:12.546185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:12.560866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:12.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:12.562751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:12.562751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:12.562751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:12.562751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 236/459] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:12.568369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:12.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 224/460] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:12.612445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95f93-d8ef-4d66-8cc2-16976c3e29e1 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.612878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95f93-d8ef-4d66-8cc2-16976c3e29e1 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.613409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95f93-d8ef-4d66-8cc2-16976c3e29e1 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.615123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 225/461] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:12.629561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.630061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.630482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.661625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:12.663077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:12.665524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:12.673304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93489a-03b4-4d3b-b583-0061f6fd4d9e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:12.689422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:12.715904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:12.718090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.718090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.718090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.718090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:12.718378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:12.720014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:12.735977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 237/462] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:12.738986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:12.746479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 226/463] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:12.754729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:12.871969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99e51-276e-4673-b5d1-a91a4c451243 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:12.875595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b24ef-fc3a-4083-8585-6ee2aa8d3b3d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:12.910168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:12.910872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.911149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.911431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.913657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:12.914458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:12.914940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:12.915404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:12.916207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:12.917881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:12.921971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:12.923434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:12.930117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:12.941769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:12.943240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:12.944101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:12.944101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:12.944101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:12.948320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 238/464] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:12.956072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:12.977586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:13.052103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03aec8-0f16-486e-a7eb-01544f221b28 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.066137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 227/465] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:12 2022] POST /identity/v3/users => generated 246 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:13.068007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.132151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.132802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.133155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.133430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.156577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.158059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:13.192010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:13.193994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:13.194853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:13.194853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:13.194853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:13.198990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 239/466] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:13.201922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.249488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00834e6-bb31-4f8b-9900-615eb24670cb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.289955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.290587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.290897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.291163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.291813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:13.292100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.293478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11516a-5995-495c-8bc6-0765eb276040 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.312484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 228/467] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:13.313728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95188c0c-d00e-4d5b-ab52-c67631842c79 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.314008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95188c0c-d00e-4d5b-ab52-c67631842c79 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.314273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95188c0c-d00e-4d5b-ab52-c67631842c79 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.315372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:13.315816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 229/468] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:13.321417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95188c0c-d00e-4d5b-ab52-c67631842c79 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.342173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d044057-0b39-4078-8b7d-39f6e7ae2b8a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.385903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.386658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.386982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.387246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.387889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:13.388188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.389632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d84f42-072d-43b3-8c3c-5ae810e3e9d2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.391590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:13.412112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 240/469] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:13.412760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.413043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.413314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.437063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:13.438457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:13.466096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95188c0c-d00e-4d5b-ab52-c67631842c79 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.508901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.509420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.509690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.509953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.511041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.512796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.526236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 241/470] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:13.527501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.598232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 230/471] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:13 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:13.600064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.611971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:13.661155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf83b6-be25-400e-9e88-77befaa8e132 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.723011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.724502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.724982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.725497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.726336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.728048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fa070-a4fa-4e22-bf2b-24eed1a60c56 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.747136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82468-a7bd-4278-8bcf-f8b2bdc6b28d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.759767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:13.785241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:13.786416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.786695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.787069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.799173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:13.800701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-18a7ec42-7cd7-4469-8e26-f7db9b9ece75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:13.817614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:13.818891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:13.819800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:13.819800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:13.819800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:13.819800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 231/472] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:13.820567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff866f3c-3f0f-4ed2-abec-e609abcac9e8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.820567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff866f3c-3f0f-4ed2-abec-e609abcac9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.823457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff866f3c-3f0f-4ed2-abec-e609abcac9e8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.826250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 232/473] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:13.829715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff866f3c-3f0f-4ed2-abec-e609abcac9e8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:13.833955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 242/474] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:13 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:13.835777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:13.836291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:13.836780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:13.859425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:13.861589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:13.947724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 243/475] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:13.970663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff866f3c-3f0f-4ed2-abec-e609abcac9e8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:13.975451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:14.018489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:14.019069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:14.019348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:14.021871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:14.021871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:14.022041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:14.023339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:14.039123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 233/476] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:14.045529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:14.052280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:14.082355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:14.107068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1b21e-6685-4caf-8563-75c204f054cf None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:14.150416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:14.151062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:14.151339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:14.151608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:14.152336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:14.153720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:14.183926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78689bb9-67f1-4c67-b333-05972c76425f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:14.198995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.sql.core [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:57:14.198995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 17:57:14.198995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [parameters: {'id': '36565eb256154ee18a2162cff8fe122e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 17:57:14.198995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80544) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:57:14.222180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:14.225612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:14.225612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:14.225612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:14.225612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:14.227399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application result.read() Aug 09 17:57:14.231638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:57:14.232500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application yield resource Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise self.value Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application session.commit() Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:57:14.237623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:57:14.238468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:57:14.240210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application result.read() Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:57:14.242253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [parameters: {'id': '36565eb256154ee18a2162cff8fe122e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:14.244201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 17:57:14.246253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.249171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 244/477] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Aug 9 17:57:13 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:14.260279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:14.357925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 234/478] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:14 2022] POST /identity/v3/users => generated 249 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:14.407236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cb493-3ed4-4dc6-aa54-e38f416d526b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:14.436173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:14.464400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:14.465093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:14.468994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:14.469257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:14.486679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-6afa5b27-e77f-4922-907f-990a6695e06b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:14.493693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:14.494645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:57:14.495477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:14.498172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 245/479] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:14.507906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:14.663873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:14.688529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce4eb5-a9de-43c7-923e-93487c6d2713 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:14.728060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:14.729063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:14.729541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:14.730023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:14.730922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:14.731437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:14.733023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b0f5a-97c8-442c-a360-c5c77c3d7619 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:14.744940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 235/480] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:15.418976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9b1f-c5b7-4c9a-9eb3-10b480184f6f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.419493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9b1f-c5b7-4c9a-9eb3-10b480184f6f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.419493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9b1f-c5b7-4c9a-9eb3-10b480184f6f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.421165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 246/481] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:15.436124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.436589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.436764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.460153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:15.461356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:15.562865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 236/482] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:15.597921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b9b1f-c5b7-4c9a-9eb3-10b480184f6f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:15.718871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b9b1f-c5b7-4c9a-9eb3-10b480184f6f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:15.754810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:15.755735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.756216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.756682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.767510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:15.773865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:15.774788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:15.775593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:15.777855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 247/483] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:15.787907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:15.895010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:15.938305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6382939-ae9e-4975-98fb-9146672ab9d2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:15.988739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:15.990156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.990525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.990820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.991492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:15.991785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:15.993167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d69f6e-6e79-4dfc-bf37-dbf808317c54 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.006881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 237/484] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:16.012449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.083810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.161911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bbb2-1b0f-4ae7-98f0-febda06f5a93 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.162225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bbb2-1b0f-4ae7-98f0-febda06f5a93 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.162502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bbb2-1b0f-4ae7-98f0-febda06f5a93 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.164249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 238/485] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:16.169534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.169816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.170093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.173537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a2d1c-a7ff-4732-9c7b-9e3cd36b4e3c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.190064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:16.191295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:16.237225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.238139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.238578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.239689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.261304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.263014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.264332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 239/486] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:16.288868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:16.292226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:16.293079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:16.293902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:16.293902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:16.293902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:16.295928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 248/487] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:16.300875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.354044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.395718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd408378-21b4-4f3f-9277-bd53dc862740 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.434741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.434741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.435000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.435254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.455870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:16.461318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:16.462202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:16.463014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:16.471295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 240/488] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:16.475619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.481718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.496167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef35648-5b92-4bbe-b2ce-3bb7489d8dde None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.541620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.542216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.542549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.542927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.543563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:16.543911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.545283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28384c9a-6d9e-4273-9f00-1dd0dbcd0dc9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.557958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 249/489] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:16.559378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3e34-a9d7-4f2b-9042-ff721f93f003 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.559664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3e34-a9d7-4f2b-9042-ff721f93f003 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.559939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3e34-a9d7-4f2b-9042-ff721f93f003 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.561312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 250/490] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:16.565991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd3e34-a9d7-4f2b-9042-ff721f93f003 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.587634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf20e-81d2-4691-b80a-287fb44adfac None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.612985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.626000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.626842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.627295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.627710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.628512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:16.628980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.631232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a1d27-60d5-4a00-a8a0-5321d8007228 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.640069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 241/491] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:16.641728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.643076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.643581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.662862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:16.664378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:16.674200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd3e34-a9d7-4f2b-9042-ff721f93f003 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.709550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.710170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.710442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.710728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.717967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.719428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:16.733042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:16.734075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:16.735006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 251/492] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:16.735585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.744051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 242/493] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:16.745171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.755560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.854970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746614-0c05-4aba-b749-cd3340801b30 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.863417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d411e-dd02-4a6c-baf5-71447c8ff6f5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.871039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.893852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.895806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.896383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.896962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.898131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:16.898744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.898868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.899495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.899900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.900277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.901234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.911767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.913343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:16.927797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:16.928706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:16.929562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:16.929562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:16.929562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:16.929562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 252/494] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:16.930307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.946009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 243/495] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:16.947329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.994359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.034206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d7b64-4b25-4edb-95bc-af9089bfb4b7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.062187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755134d4-edd9-4506-a8b8-389ffd0f42be None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.070002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.070629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.070931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.071200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.081800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:17.087714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:17.088592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:17.089396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:17.093696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 253/496] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:17.094584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.107020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.108036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.108527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.108988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.109828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:17.110461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.112186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.125032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 244/497] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:17.126216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.161895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.201939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7b51-9d28-4a2b-8e84-d253c159bf20 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.227571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.231619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a308ce1-9ccd-4e11-ad4b-e76625193034 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.241368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.241978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.242539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.242861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.266087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.266688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.266991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.267259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.268007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:17.268308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.269673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.278434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 245/498] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:17.279252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.321529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.322945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdaef0-3d13-4eb3-b305-c20acd9c66ce None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.381336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.387281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f5271-c661-4c43-822f-2e802496d874 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.423424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.424149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.424424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.424805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.433905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.435429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.441921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 254/499] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:17.443670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb21-de98-4441-8121-f714253ceff4 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.444092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb21-de98-4441-8121-f714253ceff4 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.444499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb21-de98-4441-8121-f714253ceff4 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.446283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 255/500] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:17.447126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dfb21-de98-4441-8121-f714253ceff4 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:17.452404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:17.453307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:17.454230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:17.456919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 246/501] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:17.458529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.479337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.560423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dfb21-de98-4441-8121-f714253ceff4 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.574522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097092a7-8129-40f1-b39b-30a324189206 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.594270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19928d40-577b-423f-ad70-60847394e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.594900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928d40-577b-423f-ad70-60847394e734 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.595174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928d40-577b-423f-ad70-60847394e734 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.595439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928d40-577b-423f-ad70-60847394e734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.605562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19928d40-577b-423f-ad70-60847394e734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.607078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19928d40-577b-423f-ad70-60847394e734 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.607632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.608254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.608534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.608895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.619412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.620865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-19928d40-577b-423f-ad70-60847394e734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:17.621959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:17.622903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:17.623751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:17.623751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:17.623751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:17.623751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 256/502] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:17.624529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2b8e-4b3a-4914-81b1-e2d74b0d1466 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.624766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2b8e-4b3a-4914-81b1-e2d74b0d1466 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.625031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2b8e-4b3a-4914-81b1-e2d74b0d1466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:17.640787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:17.641744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:17.642593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:17.642593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:17.642593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:17.642593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 247/503] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:17.642843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.648514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-27bd2b8e-4b3a-4914-81b1-e2d74b0d1466 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:17.649714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-27bd2b8e-4b3a-4914-81b1-e2d74b0d1466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:17.652705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.718081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 257/504] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:17.719135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee262c65-dee9-47bc-9483-7506466955eb None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.719415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee262c65-dee9-47bc-9483-7506466955eb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.719679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee262c65-dee9-47bc-9483-7506466955eb None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.722741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 258/505] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:17.722741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee262c65-dee9-47bc-9483-7506466955eb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.734481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.744809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56645794-ac73-4278-ab43-57b03014e64e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.778236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.778864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.779142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.779411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.780130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:17.780428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.781868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.799419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 248/506] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:17.800212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.835724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee262c65-dee9-47bc-9483-7506466955eb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.861296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.870258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.870888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.871163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.873550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.873550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:17.873550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.873769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2765d9-5fd3-4809-8349-f6e3abe5ef8e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.885343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 259/507] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:17.886500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.886807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.887082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.902793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b504-4b70-4265-9e7d-10570dad76fd None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.904775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:17.905937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:17.936578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.937168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.937440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.937703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.938323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:17.938621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.940022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.951940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 249/508] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:17.952932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.973802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.982124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 260/509] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:17.982962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.060647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80abed8-c187-43f8-8349-853a209bd7e4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.082015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d4d4a-2fe5-41be-be4d-9282319afef0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.084195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.096616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.096616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.096616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.096616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.107137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:18.112079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.112926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 250/510] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.114977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.120811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.121416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.121682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.121949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.182400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.189295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.194128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.216799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f6ce3-fefb-4212-99d3-957a6961ab99 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.249808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.250429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.250992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.250992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.258297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.259757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:18.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.271653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 251/511] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.272591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.294358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 261/512] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:18.295219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.319823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.375140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fbb59-1651-402e-89ae-add7575de209 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.387087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.393423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724993d-3160-4d84-a830-b97c07392f7a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.407841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.408457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.408726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.408990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.416120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.417576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.429023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.429634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.429911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:18.430005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.430942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:18.431790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:18.432468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.432561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 252/513] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.432561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.441012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.442613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:18.456828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.457784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:18.458831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:18.458831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.458831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:18.458831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 262/514] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.459229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.536802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba0ba4-2df1-499b-9fb4-de0855d7b694 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.539069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.564010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141780-3fe0-4823-b4b3-18494fb90224 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.569337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.569931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.570204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.570468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.571163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:18.571456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.572821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.580775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 253/515] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:18.586905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.597049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.597644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.598564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.598995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.599794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:18.600090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.601503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.637271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 263/516] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:18.638003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.641344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.733613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22496cfd-5142-4851-9b0c-692d77440d8c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.735978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70e623-da4c-4229-a460-d6fca6018afd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.766125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.766771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.767074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.767345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.768187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.768779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.769057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.769331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.770911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:18.771210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.772631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.778608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.780058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.790645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 264/517] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:18.791490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:18.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.795188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:18.796031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:18.796031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.796031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:18.796031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 254/518] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.796031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.814533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.858299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.894497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09722827-5d65-4682-813a-d0954ed9d02b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.896828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edac15-022a-44c1-8759-5a7a7722a925 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.928214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.928722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.928831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.929079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.929313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.929412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.929589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.929856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.930485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:18.930807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.932180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.939393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.940802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.944009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 265/519] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:18.944804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:18.958944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:18.959988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:18.961699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:18.961699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:18.961699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:18.961699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 255/520] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:18.961699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.030554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.047976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beba7ce-9473-4c1c-9375-09975262551e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.061363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b325b9-04b9-41ff-8db4-9ffcb4661c05 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.072845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.083271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b8822-72de-4691-8869-59a62934006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.083874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b8822-72de-4691-8869-59a62934006e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.084156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b8822-72de-4691-8869-59a62934006e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.084425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b8822-72de-4691-8869-59a62934006e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.096526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.097115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.097372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.097681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.099481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:19.099789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.101226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.106501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 256/521] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:19.107292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.163709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b8822-72de-4691-8869-59a62934006e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.165150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b8822-72de-4691-8869-59a62934006e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.229323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0dd64-db2a-443f-9f4c-41ef12961826 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.269685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.271175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.271458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.271725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.272368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:19.272670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.274098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.276087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.284331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 266/522] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:19.284331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b8822-72de-4691-8869-59a62934006e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.288820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 257/523] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:19.292309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.307960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.394384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b8822-72de-4691-8869-59a62934006e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.396834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c158909-c699-446b-bc2b-d216f74c3680 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.432871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.433074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f355f9-f041-4641-a782-92c4006402df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.433382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.433484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f355f9-f041-4641-a782-92c4006402df None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.433650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.433753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f355f9-f041-4641-a782-92c4006402df None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.433920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f355f9-f041-4641-a782-92c4006402df None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.434011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.434557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f355f9-f041-4641-a782-92c4006402df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:19.434649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.434886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f355f9-f041-4641-a782-92c4006402df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.436080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.436247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f355f9-f041-4641-a782-92c4006402df None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.451999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 258/524] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:19.451999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f355f9-f041-4641-a782-92c4006402df None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.509459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.528434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 267/525] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:19 2022] POST /identity/v3/users => generated 247 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:19.534178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.568718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f355f9-f041-4641-a782-92c4006402df None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.609835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.610921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.611411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.611877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.625576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.627425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.641745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:19.645448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:19.647024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:19.647877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:19.647877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:19.647877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:19.649717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 259/526] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:19.650780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.663435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6028f-41ed-430c-a48c-84d57532b166 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.702755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.703315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.703597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.703874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.712909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.714365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:19.728542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:19.729420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 268/527] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:19.731244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.781934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c54821-0153-44de-909b-19f37e1aa4d1 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.792204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.825939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.827061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.827690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.828173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.837472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.839178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:19.854384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:19.855517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:19.856599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:19.859018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 260/528] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:19.860028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.861197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75d1a-926e-4c24-9381-8e54682524c1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:19.902169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:19.903049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:19.903499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:19.903910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:19.904841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:19.905308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:19.907014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:19.916921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:19.921342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 269/529] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:19.922289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:19.966914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f213cf-a8ac-41a0-827e-77fd2484ca84 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.003182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.003789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.004098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.004370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.005054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:20.005355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.006845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.030245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 261/530] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:20.030245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.042511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6e5e-23e2-4a37-9cd4-e3e01b24f183 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.054492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:20.080140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.081025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.081447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.081874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.083694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:20.084142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.085750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.106206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 270/531] 10.222.0.36 () {56 vars in 1139 bytes} [Tue Aug 9 17:57:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:20.107449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.130500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994ad-d477-4a79-97d9-a037cb0e90ec None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.166792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.167419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.167692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.167958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.174563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:20.175294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.176941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:20.188715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:20.194727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 262/532] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:20.195678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.225054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11bc3-e1c8-40e2-857d-24759874333a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.265564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.266442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.266903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.267315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.268190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=f814ce7767d645c58a5c00a9672c54eb) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:20.268637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.272209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.287301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 271/533] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/role_assignments?user.id=f814ce7767d645c58a5c00a9672c54eb&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:20.287301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.292411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:20.318885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ba1c-2bbd-4b28-afea-aa88b472b133 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.428129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.428837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.429012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.429278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.447012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d77fe-5f15-44dc-a981-7ad253a9d15b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.477356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:20.492652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.493585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.494048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.494455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.495318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:20.495758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.497491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.517220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 272/534] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:20.527054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.554873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.556341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.638102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad8bd-f56e-4b69-ab82-62f4d70ea125 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.646278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:20.663321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 263/535] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:20.671689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.676873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.678140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.678683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.679169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.680095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=886cb33a3f254d20b25936c39b3410d8) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:20.680611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.682232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.688951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 273/536] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/role_assignments?user.id=886cb33a3f254d20b25936c39b3410d8&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:20.809152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9a0a-1cfd-46d7-89e2-5f41def28a14 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:20.857980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:20.859063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:20.859573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:20.860149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:20.876622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:20.878338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:20.897740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:20.898896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:20.899745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:20.899745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:20.899745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:20.899745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 264/537] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:20.904797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:20.986904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:21.063402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edf6a7-06d1-4a34-9fd1-6e272e841fa8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.112556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.113157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.114028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.114387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.115058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:21.115360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:21.116719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4b4a-90e9-44d3-b2df-829a190a27a7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:21.133651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 274/538] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:21.135030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.187019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:21.260783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79335b-e5d4-40e8-99d4-5a1e9a5730a0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.296014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.296631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.296902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.297172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.304688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:21.306139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:21.316481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893397-9535-48fa-8b13-5f43b55c9332 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.316841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893397-9535-48fa-8b13-5f43b55c9332 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.317193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893397-9535-48fa-8b13-5f43b55c9332 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.318820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 275/539] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:21.323010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:21.323835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:21.324669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:21.325083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.325083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.325083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.328701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 265/540] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:21.329892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.359947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:21.361146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:21.476284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:21.484418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0affd-5eb1-47c5-bc8d-a793a3471101 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.493051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 276/541] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:21.533876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.534909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.535677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.536193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.537269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:21.537810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:21.539500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:21.541116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.564183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 266/542] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:21.570983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.580049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:21.666422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dfc5c-8d9d-4606-9067-38e5700f2cfe None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.705626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.707206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.707771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.708323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.719329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:21.724803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:21.725714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:21.726490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:21.728848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 277/543] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:21.737333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.757495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372439-c3b5-49b6-9c67-18b762fc5725 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.792238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.792863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.793334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.793451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.794098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=a91c9f4d365a49d7b466292f5078fced) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:21.794427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:21.799960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:21.820707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 267/544] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/role_assignments?user.id=a91c9f4d365a49d7b466292f5078fced&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:21.868083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:21.875538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fa7d2-5e88-4bbd-a328-8a0853f9567c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:21.919892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:21.920905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:21.921386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:21.921857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:21.922698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:21.923210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:21.924786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64498705-9973-42fa-bf23-46a211556e1b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:21.934874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 278/545] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:21.940223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:21.977252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:22.020838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ef0-09c9-4879-9540-d4123953595f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.021569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ef0-09c9-4879-9540-d4123953595f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.022034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ef0-09c9-4879-9540-d4123953595f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.023984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 279/546] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:22.030826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.031460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.031920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.057871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:22.060884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:22.159154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7a06a-a4a7-4e7b-82ae-b2da320c2ce2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:22.251190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 280/547] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:22.268424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:22.269071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.269376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.271916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.281885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:22.291781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:22.292918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:22.335241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:22.336294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:22.337133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:22.337133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:22.337133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:22.343643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 268/548] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:22.348099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:22.531168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:22.535406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:22.561474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf96a-7837-4536-8851-d42dd7df4dec None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:22.592107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3d9db-d178-4ce8-8976-2e3c2ce1e5a2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:22.634495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:22.635156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.635437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.635741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.656687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:22.662168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:22.663287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:22.664674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 281/549] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:22.666201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:22.667069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.667558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.668107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.668945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:22.669438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:22.671167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:22.671706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:22.699176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 269/550] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:22.704378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:22.809375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e08a98-b3e6-431b-ae3b-6e8df38a8cd5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:22.853247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:22.854295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:22.854795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:22.855268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:22.856124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:22.856758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:22.858924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:22.876118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 282/551] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:22.882414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:22.905323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:22.948138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895984dd-c65b-4186-9bc8-ca5f3b5091da None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.018815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.020750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.024576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.025064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.030618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.042198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.046512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-93c49ea1-a31d-4b68-9f9a-3b8778e9168c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:23.065385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:23.067674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 270/552] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.068249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef338-30b1-4f04-8e21-2e46653718f8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.068249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef338-30b1-4f04-8e21-2e46653718f8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.068879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef338-30b1-4f04-8e21-2e46653718f8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.070020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 271/553] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:23.074880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b749a6-461d-49d8-aa99-ad109239684a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.075639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.075920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.076193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.104687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:23.106538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:23.114170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.115350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.116158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.116864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.130895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.132762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:23.149638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.150578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:23.152214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:23.152214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:23.152214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:23.154221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 283/554] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.155327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.183316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 272/555] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:23.183991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.237842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.280301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91484a8-d4e1-4052-a680-874858502035 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.293685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a98a3-acab-4649-a4f2-f5950603f697 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.319998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.322647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.323748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.324076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.324455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.325288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:23.325894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.327393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.330414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.331043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.333067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.333067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.333067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:23.333338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.334780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a3df1-bcd3-4f11-8d87-f0ea02a152be None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.346278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 273/556] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:23.352769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 284/557] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:23.352769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.353337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d695cba-d9a1-47c1-97b9-f4817befda45 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.353526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d695cba-d9a1-47c1-97b9-f4817befda45 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.353844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d695cba-d9a1-47c1-97b9-f4817befda45 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.355237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 274/558] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:23.359745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d695cba-d9a1-47c1-97b9-f4817befda45 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.466448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d695cba-d9a1-47c1-97b9-f4817befda45 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.468024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4f02d-7207-41d1-b107-5fa630a19d50 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.490986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.501304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.501920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.502253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.502691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.506923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.507624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.507992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.508333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.510995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.512425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.519070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:23.524327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:23.526663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-3254a16b-2df9-4612-bbf5-5d2fa7cbdd8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:23.527693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.528555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:23.529464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:23.531642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 275/559] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.531642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.531913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 285/560] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.532834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.533611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.533783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.552857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:23.554037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:23.580741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.630865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 286/561] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:23.631742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.649144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540613-229d-48e2-aa64-3fb13a113320 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.686634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.687329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.687633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.687901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.699591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.701526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:23.719597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.720578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:23.721550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:23.721550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:23.721550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:23.723194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 276/562] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.724409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.746984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c641d7c-00e9-4be8-9ed6-499fd35879dd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.757849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.782052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.782755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.783039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.783310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.784072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:23.784383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.787016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.805228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 287/563] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:23.806352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.828798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:23.837256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644ff8-16cd-4802-9163-f7d160c2e395 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.873448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.874216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.874556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.874788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.875531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:23.875831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:23.877447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:23.892210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 277/564] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:23.892210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:23.931367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c78455-fa26-4f0c-9db9-fb4db487f450 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:23.968202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:23.968827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:23.969106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:23.969374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:23.980653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:23.985295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:23.986235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:23.987140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 288/565] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:23.987869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.002211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ade6a-8ad4-4fe2-a46d-006cacd7d638 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.024363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.042239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.042889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.043172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.043606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.044629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:24.044933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.046479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.058606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 278/566] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:24.058606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.081035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.109606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fcab-d7e6-4c39-acd5-53406edeb023 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.147105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.147706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.147981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.148250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.164750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a2e85-09cb-4546-9f37-d2e8dd1f6934 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.199317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.199938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.200214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.200485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.211652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.213128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.218639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.220098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:24.226998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:24.228151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:24.229259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:24.229259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.229259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:24.229259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 279/567] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:24.229259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.248953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.296716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.320234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 289/568] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:24.321064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.326181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e7f75-b3c5-475d-a804-b042d8f6dc67 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.361859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.362370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.362644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.362935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.363715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:24.364070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.365430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77497c-9707-4863-bca8-2b305de4ad6d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.373347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 280/569] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:24.374511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d604-5739-4b26-96aa-8d0fee5c283f None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.374822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d604-5739-4b26-96aa-8d0fee5c283f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.375100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d604-5739-4b26-96aa-8d0fee5c283f None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.376316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 281/570] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:24.377005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29d604-5739-4b26-96aa-8d0fee5c283f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.424355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cf77f-da7c-409f-8b24-2b322adb163e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.450315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.458803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.466976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.466976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.466976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.467780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.469234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:24.481523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:24.482500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:24.483430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:24.483857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29d604-5739-4b26-96aa-8d0fee5c283f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.483930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 290/571] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:24.483930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.515693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.521611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.524540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.524540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.524540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.524540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:24.524540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.524978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.537290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 282/572] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:24.538089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.590867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db057a6-9cd2-42d4-8596-4e3d0a0b1832 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.624704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.625256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.625533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.625805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.635368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d751d9-bd2b-4d1b-93f6-c47f95c9dfd8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.636880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.638337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5ed919a8-00f5-4561-9dda-e25c688ed83c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:24.653840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:24.654889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:24.655946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:24.655946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.655946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:24.655946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 291/573] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:24.656391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.656653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.656920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.674429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:24.674619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.675210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.675484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.676177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:24.676393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.687041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.688343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:24.703042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:24.704152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:24.705055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:24.705055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:24.705055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:24.705055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 283/574] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:24.705055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.714754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.748251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 292/575] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:24.749142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.752896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.809181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990acef-f7dc-44cf-bf0f-a1bf8ba67f0d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.842093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.842687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.842995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.843272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.843951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:24.844254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.845714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.851708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb1f8f-053f-4324-be0e-c3afdf8761db None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.858035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 284/576] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:24.858868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.886429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.887008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.887282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.887548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.894359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:24.897112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:24.908252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:24.909342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:24.910528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 293/577] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:24.911712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:24.940720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.961487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6188d-a64d-46c9-bc59-88a745b9c542 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:24.963789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:24.996370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:24.997177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:24.997463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:24.997731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:24.998358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:24.998656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.000101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.010959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 285/578] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.011741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.011881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7174e-08b2-4243-afe0-c72954ecd632 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.046864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.047457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.047738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.048004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.048636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:25.048932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.050311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.062137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 294/579] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.062934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.112944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44855afa-185d-4c21-945e-0563fe6b340a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.149117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.149649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.149923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.150187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.153646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.161416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:25.163615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18ea3-44c2-4c16-ba4c-1761a919e2d7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:25.166471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:25.167638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:25.168504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 286/580] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:25.169182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.196355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.198082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.198748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.199027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.199290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/666454a6407846b4936d1df7c782f605` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.213409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=89d8460000674bcead45625785d81867, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.215231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.264478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 295/581] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:25 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:25.265298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.271399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a034ea-d74d-4886-a7c3-b7eba36f0300 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.311657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.312276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.312556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.312821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.313577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:25.313881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.315420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.339890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 287/582] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.339890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.397606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd3d5-d6e3-4b57-8ad6-ce41304959a4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.435834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.445249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.445873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.446151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.446625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.454441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.456234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.463059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.465803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4c882-fa8f-4801-a885-56624a3b27da None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:25.469998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:25.475451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 296/583] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:25.476556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.505837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.506525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.506835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.507110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.514508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.516020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:25.529741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:25.530741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:25.531881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:25.534837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 288/584] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:25.536648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.585550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b254b21-c13c-4fd2-b902-3caf81679003 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.640905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.641506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.641786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.642053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.642726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:25.643033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.644534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.653399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 297/585] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.654194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.686180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a46e0-d45b-400a-b2dd-7288a994a349 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.722157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.723445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.723581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.723937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.724221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.736870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.739053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.741088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:25.754756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:25.755710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:25.756610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:25.756610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:25.756610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:25.756610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 289/586] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:25.756610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.770451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f510a-2ce1-42f5-a57c-169148cc7e81 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.808138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.809102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.809388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.809660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.810659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:25.811157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.812587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.829801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 298/587] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.829801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.879408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d15f3-b306-436a-bafb-55a711e7486f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.915601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.916320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.916589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.916854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.917570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:25.917867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.919537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:25.928561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc9ce-0ade-4442-8ab5-4142d6baa7fc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:25.937888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 290/588] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:25.938883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:25.955574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:25.971633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:25.973535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:25.974113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:25.974563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:25.990562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:25.992776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.005730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:26.012612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:26.017458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:26.018325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:26.018325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:26.018325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:26.018325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 299/589] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:26.018325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.062845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd3735-b1ef-4053-9216-89be70eaa589 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.098889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.099727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.100149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.100561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.101194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:26.101489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.102883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.118304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 291/590] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:26.118304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.133092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112ba-bfad-4ce0-91ea-2597a86735fc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.167849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.168441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.168715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.168985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.169687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=2c61ea9321f94455afe35b931b9fcf8a) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:26.170170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.171438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.177416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 300/591] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/role_assignments?user.id=2c61ea9321f94455afe35b931b9fcf8a&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:26.178198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.224885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.263424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ee09-8d6b-4667-aa52-5ecbea3ddf54 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.278493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.297988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ca222-41a7-42f9-a3ac-1354357a1f19 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.303410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.304048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.304319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.304585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.316820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.318646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:26.337273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:26.338253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:26.339139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:26.339139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:26.339139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:26.339139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 292/592] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:26.339886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.340786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.341502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.341784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.342053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.342697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:26.343038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.344639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.358503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 301/593] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:26.359494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.481624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eaec9-8385-4ab4-adc1-ca45136e585b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.485173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.490622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8409ee5-a033-4809-8b8d-2a65ac58fd60 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.520671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.521291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.521563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.521828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.529484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.531061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.537885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.538967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.539384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.539791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.540588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:26.541024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.542629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:26.543979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:26.544995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:26.546074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:26.547539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 293/594] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:26.548593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.565828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 302/595] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:26.565828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.590109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.672419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a993c-61a7-4998-95f5-52c6fb3dba89 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.673285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72342-8002-4eff-9c6b-4af28458065f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.708826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.709061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.709698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.710076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.710278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.710430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.710893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.711327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.711426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:26.711710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.713209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.720522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.722242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.726642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 303/596] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:26.727353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:26.735920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:26.736836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:26.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:26.741220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 294/597] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:26.742184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.747348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.833211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ed42a-a8d7-41a1-8991-4ca88bf938c9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.845107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd742e0-e443-4dca-b24c-a5fe1474705b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:26.847035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.871276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.871863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.872137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.872403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.880013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.881450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.883900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:26.884811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:26.891891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:26.891891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/f814ce7767d645c58a5c00a9672c54eb/roles/666454a6407846b4936d1df7c782f605` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:26.893152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:26.894097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:26.895756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:26.899046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 304/598] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:26.899934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:26.905923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=f814ce7767d645c58a5c00a9672c54eb, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:26.907633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:26.982100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:26.984580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 295/599] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:26 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/f814ce7767d645c58a5c00a9672c54eb/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:26.985804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.041177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0500-20a1-42dd-a829-6fb17eb576a8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.124273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f180-b65a-45d1-ae7a-24619baa95ca None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.133754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.134374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.134647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.134954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.135687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:27.135987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.137422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.164178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 305/600] 10.222.0.36 () {56 vars in 1139 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:27.170257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.170762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.171373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.171647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.171937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.172609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:27.172903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.174337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.187698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 296/601] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:27.188295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:27.196383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.357735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1505ec4-bca2-4106-97f1-d3bf3ca9cbb8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.373169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a769d-1b0c-4ac6-8fbc-6c7d605d20ce None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.426453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.427117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.427391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.427654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.432530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:27.440064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.440725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.441046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.441340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/886cb33a3f254d20b25936c39b3410d8/roles/666454a6407846b4936d1df7c782f605` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.444582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.446942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f433b878-f869-4163-9a8a-fb185ea0f142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:27.471911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:27.473392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:27.474670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:27.474670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:27.474670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:27.474883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=886cb33a3f254d20b25936c39b3410d8, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.474967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 306/602] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:27.475049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8edf01-466c-42a7-96b8-e11c50de81f5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.475299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2173-e166-4fd0-980d-a644b8a2443c None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.477245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2173-e166-4fd0-980d-a644b8a2443c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.477245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2173-e166-4fd0-980d-a644b8a2443c None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.480054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 307/603] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:27.481691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a2173-e166-4fd0-980d-a644b8a2443c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.546105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 297/604] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:27 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/886cb33a3f254d20b25936c39b3410d8/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:27.555534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.555873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.556155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.591647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:27.592849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:27.609893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a2173-e166-4fd0-980d-a644b8a2443c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.653194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.654403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.655304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.655780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.656656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:27.657153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.659248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.671858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:27.676828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 308/605] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:27.682866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.763743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:27.764452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 298/606] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:27.799002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.815792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbb6a8-57a3-4489-bd28-f10d22a8822d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.859652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.860902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.861738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.862480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.871192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:27.872643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:27.885564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:27.886468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:27.887390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 309/607] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:27.891777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:27.923188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83407e-6090-4724-b04e-d61891b917b3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:27.967344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:27.967961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:27.968230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:27.968491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:27.985159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:27.991757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:27.992725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:27.993558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:27.996371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 299/608] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:28.002739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.027204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcb99-5dad-445e-b91d-b2769412260a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.038894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.077967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.087546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.088203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.088479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.088798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.089501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:28.089803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:28.091266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:28.114229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 310/609] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:28.119005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.208875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee999c-a500-4382-b69a-9f9a3390dff2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.250417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.251073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.251347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.252429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.253126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:28.253660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:28.255118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:28.265617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 300/610] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:28.272097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.306195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b744b3-30a4-47b9-bc5a-34ccac20b35f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.345124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.346119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.346543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.346993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/a91c9f4d365a49d7b466292f5078fced/roles/666454a6407846b4936d1df7c782f605` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.365930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=a91c9f4d365a49d7b466292f5078fced, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:28.367654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:28.430376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.436948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eeb27-cc70-4b5e-bb7b-e5228c8c79ca None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.460884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 311/611] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:28 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/a91c9f4d365a49d7b466292f5078fced/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:28.483269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.484273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.484711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.485201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.498369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:28.500383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-0edccacf-e982-478d-9289-bb4eb8f7ebd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:28.518223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:28.519440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:28.520653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:28.520653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:28.520653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:28.522518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 301/612] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:28.528344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.543403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044345ad-a886-4003-9b6c-cb4125f0e2f6 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.543914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044345ad-a886-4003-9b6c-cb4125f0e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.544331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044345ad-a886-4003-9b6c-cb4125f0e2f6 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.546806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 302/613] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:28.552530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.559567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.559567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.563118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.591649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:28.594666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:28.710843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e13619-abc2-4f48-b618-ace7ef385c7c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.720566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 303/614] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:28.752537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.752750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.754002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.754483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.754942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.755731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:28.756170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:28.757688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:28.771755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 312/615] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:28.777485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.853149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.877020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e407a5-a384-4196-8c1a-b07015fcbf69 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:28.919508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:28.920247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:28.920532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:28.920804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:28.945435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.947614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:28.958733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:28.967479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 304/616] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:28.968333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:28.969080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1be21-bb27-4856-a623-a2f9232e5cce None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.048149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.048816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.049050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.049316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.061727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.064005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:29.086286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:29.088886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:29.092196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 313/617] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:29.093670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.163476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24f9c8-4361-4b7f-9ce5-0cdcb9725c98 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.216703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.218325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.218831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.219311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.220165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:29.220658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.223111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.232307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 305/618] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:29.232681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.243890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.252261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ebc60-0147-4e1e-8c5b-c9c803e01af7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.288703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.289311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.289588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.289908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.290635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:29.290967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.292404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c5a08-5407-4280-975b-764f50ec31cf None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.310796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 314/619] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:29.311170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.318765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccdac7-9de6-41c2-aeb1-7a34b2a23a6a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.318765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccdac7-9de6-41c2-aeb1-7a34b2a23a6a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.318765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccdac7-9de6-41c2-aeb1-7a34b2a23a6a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.318765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 315/620] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:29.318765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccdac7-9de6-41c2-aeb1-7a34b2a23a6a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.365643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ed113-0372-49b3-adfa-7211f35a0f78 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.401161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.402061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.402484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.402925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.414607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.416281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.429531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccdac7-9de6-41c2-aeb1-7a34b2a23a6a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-dd3ddaad-a37a-4ca5-b05c-e2764a10536f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:29.431619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:29.432724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:29.433800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:29.433800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:29.433800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:29.435623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 306/621] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:29.436950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.437374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.437849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.457674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:29.459186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:29.483544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.484179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.484460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.484738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.512548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.539013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 307/622] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:29.539925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.555058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.561926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.563456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.676056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8274c1-ea18-4727-8906-b8b8de6f03a9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.676790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 316/623] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:29.677688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.720742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.721670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.722996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.723108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.723673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:29.723976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.725945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.746962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 308/624] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:29.747849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.794382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.799160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:29.801362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc232ee1-206d-40d9-b15b-9de99e37896c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.837850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.838638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.838996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.839325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.851543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:29.857024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:29.858191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:29.859266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:29.866405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 317/625] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:29.867578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.870201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4384d-060c-4de4-9e9a-ad0ffa73aebf None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:29.916650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:29.917457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:29.917830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:29.918194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:29.931652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:29.933322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:29.948789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:29.949804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:29.950736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:29.950736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:29.950736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:29.954658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 309/626] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:29.955461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:29.993028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2510c3-92ee-48f0-8338-cfd5ca335a1a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.032134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.033055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.033536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.034616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.043788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.045418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.053702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-e4994f04-2333-491e-a16c-c0b61ab51451 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:30.058230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:30.059545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 318/627] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:30.060532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c2c7-ead2-44c5-8d71-6ddb9eeab2da None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.061182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c2c7-ead2-44c5-8d71-6ddb9eeab2da None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.061182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c2c7-ead2-44c5-8d71-6ddb9eeab2da None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.062335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 319/628] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:30.063054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c2c7-ead2-44c5-8d71-6ddb9eeab2da None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.069557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f6cc-c7f6-4aed-8f91-95f7e8bf0b3d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.083159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.107507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.107963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.108240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.108534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.109162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.109491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.110894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03139269-7c74-4f78-8a1b-3a5ae6128707 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.120735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 310/629] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.123214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.123513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.123781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.141633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:30.142829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:30.176620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c2c7-ead2-44c5-8d71-6ddb9eeab2da None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.211800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 311/630] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:30.212466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.216478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.217446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.217948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.218423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.219341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.219925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.222040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.241494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 320/631] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.242391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.314205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.327877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.329222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598a5ba-aa5d-461e-aa79-0f8f03ccb392 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.351665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa16f-8e7a-432f-bae3-f071fe4dd9a9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.365753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.366365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.366649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.366947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.367671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.367973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.369408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.388711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.388906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 312/632] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.389649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.390047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.390392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.390743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.405859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.407346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:30.423108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:30.424042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:30.424923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:30.424923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:30.424923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:30.424923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 321/633] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:30.425154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.522961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468189a-68b0-447c-843f-b0f12c844d19 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.549436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c741a-6f95-4a2d-9219-91dca0093c53 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.558587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.559238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.559517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.559782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.569899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:30.574520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:30.575477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:30.576318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 313/634] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:30.577538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.583296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.583885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.591733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.592438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.592850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.593127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.600631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.602096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:30.613499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:30.614440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:30.615379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 322/635] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:30.615890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.688683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d66b-5ba5-4c15-88a7-6021f1b1c331 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.725506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.726136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.726420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.726689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.734989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d74a-dde3-45ba-856f-61b9c309fb41 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.737301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.738892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:30.754526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:30.755552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:30.756435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:30.756435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:30.756435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:30.760429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 314/636] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:30.761265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.773317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.774286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.774568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.775200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.775846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.776148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.777576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.790631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 323/637] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.791321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.825220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.843247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:30.868951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fb20-2df6-44a4-a5fb-af0437cd10fc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.893173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a42d-9c24-4e2f-a1e9-a22cd73b77c3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:30.902611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.903252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.903528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.903798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.904452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.904747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.906117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.914221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 315/638] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.914871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:30.927054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:30.927657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:30.927930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:30.928196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:30.928875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:30.929174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:30.930578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:30.942332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 324/639] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:30.943140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.012782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096f879-587c-489a-a296-9d5f921d0c7d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.028955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.043683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13dbf-18db-4e23-a2a4-39db7d3f5df9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.045670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.046268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.046547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.046850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.047486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:31.047783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.049144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa1c18-c707-4e12-acf7-6c77e1894a4e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.053320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.061416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 316/640] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:31.062685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bad6b-f857-428e-a64a-cb9a79f7eaa8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.062996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bad6b-f857-428e-a64a-cb9a79f7eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.063262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bad6b-f857-428e-a64a-cb9a79f7eaa8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.064587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 317/641] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:31.065189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bad6b-f857-428e-a64a-cb9a79f7eaa8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.081151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.081929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.082211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.082567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.090207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.091922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:31.103333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:31.104341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 325/642] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:31.105286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.168988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bad6b-f857-428e-a64a-cb9a79f7eaa8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.203168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.203785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.204059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.204324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.205486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910227f8-37f8-46f1-8296-76f31a62a22a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.214290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.215766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:31.230278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:31.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:31.232525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:31.232525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:31.232525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:31.232525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 318/643] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:31.232525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.241887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.242489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.242804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.243431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.244162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:31.244788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.246229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fd366-473a-4262-bdbd-774c14522bf5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.251927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 326/644] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:31.253041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.253425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.254095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.268171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.279144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:31.280471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:31.311284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.344825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70d8e-ba3a-4ba7-823c-0b4df5e4f4fa None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.370733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 327/645] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:31.370733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.382418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.383239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.383609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.383979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.391935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.393397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:31.406277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:31.407256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:31.408166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 319/646] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:31.408645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.491763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc946122-e26e-4b01-957f-ef676dc6455a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.521640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.526892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.527500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.527778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.528045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.528775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:31.529072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.530537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.533083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4998a-7cf5-4bcb-a30e-1b8d8a8f8540 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.548141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 328/647] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:31.548985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.571025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.575304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.575906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.576178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.576439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.577074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:31.577366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.578760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.596558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 320/648] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:31.596558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.656245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bf861-30dd-4d4a-b07f-cde1411bd6ad None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.692384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.693011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.693287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.693563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.699237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f7171-c165-4ebc-b4c9-2b3bbfaca3c6 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.703426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:31.708088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:31.709031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:31.709877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 329/649] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:31.710578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.741248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.742291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.742771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.743287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.744143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:31.744616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.746442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.758968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 321/650] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:31.759845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:31.776715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.804505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:31.830071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d622e9-43d6-47b8-99c3-f646adfd6f65 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.865900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.866417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.866692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.866989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.873027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a56e3-481f-4806-a90b-32acc4be99c4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:31.910369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:31.911299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:31.911868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:31.912438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:31.920225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.921963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.932979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:31.934008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:31.935144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:57:31.936083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:31.936656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:31.939885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 322/651] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:31.940598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.014082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.027160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.047458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d015c9-60b5-4d92-a1ca-9cb25ec72e34 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.052491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 330/652] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:32.053546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.087277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.087891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.088173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.088440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.089099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:32.089401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.090794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.101636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 323/653] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:32.102451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.165492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1aee2-c719-475f-bfa2-cdf2082bf15e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.203195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.203840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.204231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.204504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/2c61ea9321f94455afe35b931b9fcf8a/roles/666454a6407846b4936d1df7c782f605` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.208272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9e336-df5e-4343-9adb-8296466a0232 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.220277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=2c61ea9321f94455afe35b931b9fcf8a, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.222670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.236799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.241560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.242152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.242426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.242695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.243478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:32.243781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.245197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.261766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 324/654] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:32.262656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.290735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 331/655] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:32 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/2c61ea9321f94455afe35b931b9fcf8a/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:32.290735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.332325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.399211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963a32c-d8d8-41a7-9d94-3227eacca927 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.430791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f968d24-16c1-4441-ae32-31308a8ec6ec None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.439998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.441355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.441728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.442112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.457060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.458910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.478797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.479437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.479718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.479986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:32.481630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:32.482534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:32.484554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:32.484554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:32.484554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:32.484554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 325/656] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:32.484554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.494671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.496326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:32.513491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:32.514631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:32.515577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:32.515577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:32.515577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:32.518874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 332/657] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:32.520811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.542683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.610002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2071-e11e-4673-97f5-ca03e8987c47 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.634217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.654091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e022e10-3155-43a9-85bf-fcfe80be7843 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.659251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.660552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.660981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.661424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.692936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.693629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.694017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.694384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.695639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:32.696039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.697607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.716281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 333/658] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:32.716281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.761484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.764101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.826148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:32.833142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0938f6-2a02-49f8-9b5a-78800d4c65f7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:32.877017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:32.877779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:32.878190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:32.878734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:32.879554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:32.879950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:32.881632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:32.891633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 326/659] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:32.892674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.899961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 334/660] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:32.900988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:32.904039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.007977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5d86e-8087-4b7a-9b7d-df83b4af9c70 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.029259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d768-6157-4a73-bb25-a8394060d04e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.045567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.046533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.047030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.047537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.055481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.057247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:33.070346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:33.071477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:33.072350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:33.074682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.075164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 327/661] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:33.076358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.079970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.082165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.082696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.083088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.098942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.101535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:33.119169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:33.120110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:33.121026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:33.121026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:33.121026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:33.123219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 335/662] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:33.124373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.214782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45248e1e-9ad5-46f5-9a4e-ef2cae3063e7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.239779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a845d-0937-4170-b57e-4a358d869f10 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.265417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.265963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.266234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.266496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.274955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.276379 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.281245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.282108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.282918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.283514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.284909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:33.285803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.287434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:33.289497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:33.290693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:57:33.292038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:33.297985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 328/663] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:33.298968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.312487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 336/664] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:33.312487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.383013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.430301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e6a98-3a91-48a8-a2fa-f2aedd57b94d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.430981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232e6f-9acc-4e24-84f1-6fe8434a4841 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.445997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.471133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.471752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.472647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.472943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.473834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:33.474080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.475506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.475651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.476134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.476412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.476684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.477701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:33.478009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.480114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47cec-4286-4fbe-a751-ef2626e63471 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.493699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 329/665] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:33.493699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 337/666] 10.222.0.36 () {56 vars in 1139 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b70f-df94-49bc-be23-a947a068f531 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b70f-df94-49bc-be23-a947a068f531 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b70f-df94-49bc-be23-a947a068f531 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 338/667] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:33.510664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b70f-df94-49bc-be23-a947a068f531 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.606129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce4ce-52b0-4323-a5d6-33bd7d578c16 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.623611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b70f-df94-49bc-be23-a947a068f531 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.630894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.642382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.645723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.645723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.645723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.645723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=f814ce7767d645c58a5c00a9672c54eb) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:33.646114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.647283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.653459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 330/668] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/role_assignments?user.id=f814ce7767d645c58a5c00a9672c54eb&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:33.654332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.664241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.665615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.666119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.668346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.677972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.680950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-318f5cd0-2046-458d-b8a4-c388950c268c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:33.697852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:33.698878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:33.699796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:33.702886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 339/669] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:33.704593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.705108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.705786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.730698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:33.733106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:33.770617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.813868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db66d-46f5-4af7-91bd-07e2e5ddf916 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:33.834648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 340/670] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:33.834916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:33.861078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:33.861624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:33.861899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:33.862164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:33.938661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:33.953066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:33.958277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:33.963486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cf691-3f94-41a6-878d-41fad6279f84 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.016170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.017442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.017928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.018631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.020617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:34.021137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.023348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.047205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 341/671] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:34.047820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.078806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 331/672] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:34.078806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.087155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.180408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c390-a8b0-425e-87d5-3f0017af9315 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.181572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.199851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a20f-1fcc-41ab-a4aa-f9683f01beba None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.219689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.220906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.221331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.221751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.232651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:34.239203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:34.240173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:34.241118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:34.246869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 342/673] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:34.246869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.247768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.248167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.248438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.248697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.249414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=886cb33a3f254d20b25936c39b3410d8) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:34.249711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.251195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.262850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 332/674] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/role_assignments?user.id=886cb33a3f254d20b25936c39b3410d8&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:34.264113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.408799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1b2a9-43c6-4714-83ee-0c2a482980e3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.417523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.440187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb63b-40e4-43ee-a878-79dcb3562063 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.458037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.458734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.459055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.459417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.460701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:34.460701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.461739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b364a50-1ec8-4423-835e-56800ce24f97 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.474342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 333/675] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:34.474342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a871b-cdbd-4d2a-b313-fe73c5a839f1 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.474342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a871b-cdbd-4d2a-b313-fe73c5a839f1 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.474342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a871b-cdbd-4d2a-b313-fe73c5a839f1 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.474342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 334/676] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:34.480316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.480602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.480876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.502225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.502899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.503234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.503713 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.505881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:34.507141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:34.514464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.515969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.517177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:34.532657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:34.533692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:34.535309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:34.535309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:34.535309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:34.535309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 343/677] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:34.535585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.606876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 335/678] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:34.608829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.661480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f918b-d5f5-4121-a405-036c7cd16ccb None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.704489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.705493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.705966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.707117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.710207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.723482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.726311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.735930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477c291-9270-48ef-a8fd-fbf879457d22 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:34.745574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:34.746524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:34.748053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:34.748053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:34.748053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:34.750585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 344/679] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:34.751913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.779349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.780524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.780967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.781390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.782170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:34.782614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.785772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.806824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 336/680] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:34.807918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.810890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:34.870915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c79a-053d-49f3-b994-9abca26e4770 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.910444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.911493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.911876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.912248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.917009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f0c64-bc61-4442-a110-910f2ef44ff2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:34.923618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:34.929583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:34.930626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:34.931504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:34.936685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 345/681] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:34.937608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.952997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:34.953594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:34.953921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:34.954195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:34.955272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:34.955509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:34.957832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:34.975649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 337/682] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:34.976569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:34.999116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.041646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.050795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6282638-ff92-4e4b-b564-601a9f95e88d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.082779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b112-0dfa-4dc9-add2-163a14f4ee02 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.089383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.090306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.090945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.091433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.100082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.101990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:35.115939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:35.116936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:35.117823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:35.119395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.119995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.120267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.120609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.121063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 346/683] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:35.121257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:35.121680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.122489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.123081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.134779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 338/684] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:35.134779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.235774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54927-7535-4479-bf9b-26a38f6a5112 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.239282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9a50-b949-488a-b5d7-eac68fbd0dd0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.247159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.261382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.273589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.274272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.274550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.274873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.277287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:35.277287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.277472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.277585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.277820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.278191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.278554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.287911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.289509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.290954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 339/685] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:35.291779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:35.302337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:35.304076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 347/686] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:35.306164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.398898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891c6d-f513-4b4e-a6ab-45c3d8d8e19c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.415291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f03138-0d10-4ace-b993-e5e3b333ed34 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.432726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.433345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.433618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.433888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.444685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.446132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.451106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.451705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.451981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.452251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.453120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=a91c9f4d365a49d7b466292f5078fced) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:35.453719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.457368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.462450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 348/687] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/role_assignments?user.id=a91c9f4d365a49d7b466292f5078fced&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:35.469970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:35.470929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:35.473766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:35.476022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:35.476022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:35.476022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:35.476022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 340/688] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:35.476022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.515243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.533672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.607859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef63b3-06ea-41f3-9f08-ff89e6416bc3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.618735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec860abe-7b02-4b51-97b2-7c4df30990a0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.652197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.653089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.653219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.653477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.654275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:35.654491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.659809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.677922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 349/689] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:35.679918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.680755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.681127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.681490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.682417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:35.682851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.685115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.694099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.718413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 341/690] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:35.731928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.810394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.824178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346da06-e608-4a4f-a7d8-615b050fade3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.833765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:35.841577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab2ac-1885-4d78-87c2-01f3a0e88279 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:35.865156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.866112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.866112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.866404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.882230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:35.883216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:35.883691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:35.884164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:35.892232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.894150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:35.908090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:35.909066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:35.910233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:35.913058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 342/691] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:35.920339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:35.950020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:35.952327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997a353-96f3-4cec-9bd7-c4cf09a62f86 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.052870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd83d2-5b2d-4cb7-bae7-13b52ebc7524 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.068153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.083070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 350/692] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:57:36.084706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabd71-f126-4ca4-99b7-b8f7ecc41410 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.084994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabd71-f126-4ca4-99b7-b8f7ecc41410 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.085268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eabd71-f126-4ca4-99b7-b8f7ecc41410 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.086674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 351/693] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:36.089858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eabd71-f126-4ca4-99b7-b8f7ecc41410 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.092595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.093745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.094898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.095202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.095961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:36.096265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.098133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33685935-0d9e-4618-b926-dd3ab5d7a7a4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.116856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 343/694] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:36.118239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.118435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.118694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.138395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:36.139654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:36.150845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.206845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eabd71-f126-4ca4-99b7-b8f7ecc41410 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.222727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 344/695] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:36.222727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.268557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.270243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.270748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.271220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.282614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.284915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:36.300682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:36.301765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:36.302661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:36.302661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.302661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:36.304690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 352/696] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:36.305730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.329513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bfea4-d56e-435a-bd42-64c1922ebf69 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.360168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.375649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.376988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.377419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.378235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.397359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.424623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7db61-50b2-4e44-9758-4b9891dc2e3f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.459260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.460084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.460463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.460848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.469295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.470961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.588400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 345/697] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:36.589550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.594738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.596094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:36.611581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:36.613589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:36.614671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:36.614671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.614671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:36.620597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 353/698] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:36.621741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.648270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.713271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1661d4-edc9-4305-875d-b90989bf9764 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.730668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.743269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca23d-52c6-4f27-b210-3130ec70aad3 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.751761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.752367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.752636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.752944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.753653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:36.753961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.755471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.766790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 346/699] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:36.767576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.782757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.783393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.783660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.783927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.794979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.796414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-cdc8cacd-59ab-496f-b02a-8ba5e48a6f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:36.810968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:36.812096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:36.812981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:36.812981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:36.812981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:36.812981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 354/700] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:36.813216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d895-584c-4ab6-8410-f2f2a3b337bd None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.813638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d895-584c-4ab6-8410-f2f2a3b337bd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.813734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d895-584c-4ab6-8410-f2f2a3b337bd None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.819169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 355/701] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:36.819169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21d895-584c-4ab6-8410-f2f2a3b337bd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.879533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c08aa-0ff9-4c4f-b1c6-ee83c3c014a0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.882737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:36.917597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.918198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.918469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.918757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.919390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:36.919694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.921159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:36.925426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21d895-584c-4ab6-8410-f2f2a3b337bd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:36.934526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 347/702] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:36.936460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:36.969154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:36.969841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:36.970123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:36.970388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:36.981416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:36.983629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5aa4e2e6-836b-461a-999b-6cde7c18ce69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:37.000336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:37.001306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:37.002177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:37.002610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 356/703] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:37.003597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.003876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.004143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.012926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.026466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:37.028349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:37.045281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84299-dc3c-4169-81f0-e2438237b774 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.082571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.083226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.083503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.083772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.084403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:37.084701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.086072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.100034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 348/704] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:37.100807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.111341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 357/705] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:37.112155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.141107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.218839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096c56-a30a-4670-bcc7-fd7845e72323 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.222110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe386b7c-255f-4193-8ef9-27f3b9e9f560 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.251799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.252398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.252673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.253032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.253793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.255263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.259294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.259990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.260382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.260753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.261538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:37.261938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.263576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.265555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.276802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 358/706] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:37.277593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.366738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 349/707] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:57:37 2022] POST /identity/v3/users => generated 254 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:37.366738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.393267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd79b7-2105-4a42-8cb8-971405d247f2 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.442651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.443454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.443791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.444123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.453932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.455476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.463639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:37.478408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:37.479646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:37.480640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:37.484188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 359/708] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:37.484188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.505340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1758e-924b-4cba-ae8b-169c3c2eb694 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.552781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.557592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.558235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.558507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.559550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.559550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.560740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:37.574869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:37.575941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:37.576916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 350/709] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:37.577775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.618728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ee4c8-08ac-4c90-91cf-170fb7c3c2ed None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.665122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.666140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.666419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.666736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.667482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:37.667845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.669281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.677399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 360/710] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:37.678695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.717498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026eb7-3254-4fe0-badf-b7eab68f5721 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.759071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.759685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.759965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.760232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.760994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:37.761293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.762789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.776803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 351/711] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:37.789773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:37.795036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.901471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb79da-5763-486c-af6b-40cf2e96807d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.940113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:37.941629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:37.941915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:37.942189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:37.942857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:37.943465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:37.946892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:37.947665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:37.957327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 361/712] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:37.959277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9e464-8577-40f4-8180-bb981edc9f21 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:37.963469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:38.004240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:38.004838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.005108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.005380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.006101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=2c61ea9321f94455afe35b931b9fcf8a) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:38.006404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:38.007882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaed859-bb42-4578-b04c-1b7173d0508b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:38.016830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 352/713] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/role_assignments?user.id=2c61ea9321f94455afe35b931b9fcf8a&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:38.018048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e03fb4-6d2a-497f-be46-7eb5a5a92f72 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.018375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e03fb4-6d2a-497f-be46-7eb5a5a92f72 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.019813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e03fb4-6d2a-497f-be46-7eb5a5a92f72 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.025144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 353/714] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:38.026856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.027143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.027413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.046879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:38.047966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:38.120107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fdb83-e8e2-4adb-9734-94b800df3edd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:38.171137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:38.171873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.172188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.172468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.173110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:38.173728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:38.175996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:38.183406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 362/715] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:38.197170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:38.203686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 354/716] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:38.240884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:38.297731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:38.362279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18eb9e-3997-409a-8e59-4d6805389ff5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:38.401782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:38.402406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.402679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.402973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.403545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:38.404892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:38.416717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:38.417741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:38.418614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:38.422731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 363/717] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:38 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:38.429146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:38.547850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34293c-60b7-4bee-b63e-b7f293a8cab9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:38.586467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:38.587154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.587433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.587700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.588384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] RBAC: Adding query filter params (name=network) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:38.588687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:38.590123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2acd-6515-49d6-b4ac-057978c22a0b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:38.595240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 355/718] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:38.601415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:38.754456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493b9f-045a-42a5-aa79-5ef9084d8ae5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:38.772202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:38.802927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:38.803566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:38.803846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:38.804110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:38.804736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] RBAC: Adding query filter params (type=network) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:38.805030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:38.806373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670642-4fca-4e42-ae68-f36ae9cd1728 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:38.814633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 364/719] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:39.763959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f036d6-9afe-4916-a108-1e6db312e8e2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:39.765013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f036d6-9afe-4916-a108-1e6db312e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:39.765504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f036d6-9afe-4916-a108-1e6db312e8e2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:39.767306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 356/720] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:39.773781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:39.774632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:39.775143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:39.795123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:39.796617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:39.880256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 365/721] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:57:39.916120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f036d6-9afe-4916-a108-1e6db312e8e2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.011204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.063561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f036d6-9afe-4916-a108-1e6db312e8e2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.109680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.110301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.110572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.110861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.130691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c795deb5-5c64-4fcf-ac62-f080bbfd4223 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:40.135816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:40.136746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:40.137570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 357/722] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:40.144906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.281761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927fcf-3669-4c35-b1d1-40d85478123a None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.282021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927fcf-3669-4c35-b1d1-40d85478123a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.282428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927fcf-3669-4c35-b1d1-40d85478123a None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.283637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 358/723] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:40.290950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.291527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.291999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.293735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278acdc-e413-494d-abae-8f0421de90a2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.296424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.328228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:40.330298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:40.347459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.348078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.348355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.348621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.349281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:40.352196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.354743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.378895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 366/724] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:40.385361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.468393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 359/725] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:40.500644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.604551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ffd27-de4b-4bf5-82af-35c81484341f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.613811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.675656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.676434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.676709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.676973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.691422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728db416-cf9e-4983-95f8-1c33eea99bd0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.702063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.707510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:40.734385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:40.735670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:40.737270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:40.737270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:40.737270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:40.737270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 367/726] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:40.744491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.745144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.745419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.745688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.747039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.768038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:40.774182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:40.776305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:40.779102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 360/727] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:40.786746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.811099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.919685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346c60-6675-4fd6-b767-3d6ef6356244 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.920239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f8cce-38d7-4913-a97c-d1d01234127e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.939694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.963791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.964231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.964426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.964660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.964840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.965130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.965398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.966056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:40.966350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.966620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.967247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:40.967546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.967767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21941a0-3d2a-41e4-8b5d-48d49dceb280 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.968899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59847f-7d9b-430d-866d-2a3d46f2613e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.982794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 361/728] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:40.984083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623d5c-52d4-471f-816f-a09183755884 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.984364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623d5c-52d4-471f-816f-a09183755884 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.984631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623d5c-52d4-471f-816f-a09183755884 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 362/729] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b810d-d32f-4691-a043-b750815e3ad8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b810d-d32f-4691-a043-b750815e3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b810d-d32f-4691-a043-b750815e3ad8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 363/730] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:40.991093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b810d-d32f-4691-a043-b750815e3ad8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.994509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 368/731] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:40.995903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.996185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.996509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.014763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:41.015937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:41.106736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 369/732] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:41.106736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.117929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.118714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b810d-d32f-4691-a043-b750815e3ad8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.156608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.157344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.157683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.157959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.171190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.172557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e07284a8-16a3-448c-9b38-3f0cba1a3f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:41.187710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:41.188716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:41.189676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:41.189676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:41.189676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:41.189676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 364/733] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:41.190228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.190455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.190738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.210378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:41.211668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:41.227323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ac80c-29d6-4539-8ac6-6a2b81a1dc34 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.233167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.266067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.266719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.267007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.267280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.276348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.278253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.293749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 365/734] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:41.293749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:41.294293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:41.295179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 370/735] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:41.296110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.369536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.395648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de75680-9f20-4a3c-a700-8ee9ebd76a82 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.407157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a690f-36bf-49b4-98e1-cf69419ef33a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.432384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.432923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.433192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.433454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.443295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.443901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.444182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.444455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.444618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:41.445094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:41.445396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.446769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:41.449237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:41.450171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 366/736] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:41.451068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.462746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 371/737] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:41.464042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.485198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.567977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c00cbf-1647-4c41-baa5-7f9180a0564e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.595863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.604823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.605438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.605718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.605991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.606641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.608051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.676991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 372/738] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:57:41 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:41.677830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.697084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be2872-aa02-4783-91e4-373fd079a2df None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.741966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.742587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.742897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.743176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.743913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:41.744217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.745680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.764774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 367/739] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:41.766051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.811661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a0198-6b4e-49af-99b2-0bab845734a2 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.845273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.852279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.852894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.853172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.853447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.854107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:41.854408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.855806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa60bc-22b4-43ad-bce9-66eb87c66448 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.866856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 373/740] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:41.868351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027ff3-4845-439d-9ea1-9733394fd3b5 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.868842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027ff3-4845-439d-9ea1-9733394fd3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.869262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027ff3-4845-439d-9ea1-9733394fd3b5 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.870907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 374/741] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:41.871777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027ff3-4845-439d-9ea1-9733394fd3b5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.882389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350df659-30a9-4836-87db-acf39eb21aca None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.921024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.921980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.922507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.922803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.925947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.930466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.931930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:41.943496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:41.944690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:41.945879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:41.954060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 368/742] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:41.954060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.983398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027ff3-4845-439d-9ea1-9733394fd3b5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.019907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.020522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.020868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.021143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.066130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4d199-1d9b-4b63-8d14-5818b9e4fbe7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.092688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.094129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.096557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.103377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.103962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.104237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.104502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.115836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.117699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8ad98ccc-a515-4243-bb90-7da9d0182aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:42.132994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:42.133984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:42.135380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:42.135380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.135380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:42.135380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 369/743] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:42.135750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.136029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.136301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.152829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.157023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:42.158187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.209930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 375/744] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:57:42.210739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.233893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 370/745] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:57:42.235094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.323677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab7dd-832b-4b39-a264-dcbba1375caa None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.358365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.359027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.359308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.359581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.360300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:42.360602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.361595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744c95f-80d1-47cd-8f05-ede39938f393 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.362084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.376089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.386145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 376/746] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:42.386145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.398403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.402035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.402649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.402970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.403243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.403882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:42.404181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.405549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.422869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 371/747] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:42.423591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.500877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab2202-2924-4061-895d-d11672216db0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.526228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23647ea1-aedb-4ebe-a79b-df9986cc35d8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.538040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.539025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.539452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.539882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.550679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.552156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.560225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.560986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.561260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.561524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.562151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.563537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:42.567461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:42.568362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:42.569234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:42.569234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.569234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:42.569234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 377/748] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:42.569471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.629703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.648452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 372/749] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/services => generated 240 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:42.649176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.687955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.698738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6661a6-841f-4338-895a-6f8a29f33cc1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.769787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.771127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.771613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.772413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.780185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250e8c-69c2-439f-9b1b-1e1c8e3f1b06 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.789478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.791016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:42.808867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:42.809873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:42.810784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:42.810784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:42.810784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:42.811400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 378/750] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:42.816123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.828961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.830251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.830897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.831848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.843620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.845198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:42.859429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:42.860854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 373/751] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:42.861811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.943673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ec911-46a1-41b4-83bb-e28051536c1d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.953110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.985892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.991778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.992773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.993059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.993596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.994250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:42.996402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.996518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82ef64-fa37-48d1-a2da-7d39e8161021 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.996613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.014803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 379/752] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.015744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.037931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.038505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.038803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.039073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.039765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.040160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.041651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.054397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 374/753] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.055277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.139880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35027-f8cc-47e5-aa27-18c54eff0af6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.164501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8a10-1999-4cf7-8d8f-43b39366ef07 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.176964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.177585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.177862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.178135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.179015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.179421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.180944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.200809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 380/754] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.201868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.202792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.203417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.203688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.203952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.212607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.214242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:43.226649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:43.228824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 375/755] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:43.230971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.239044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.246999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.325146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc321bde-9442-4988-b484-35ebddc73348 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.336243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c1fa6-a8d9-4bf4-9149-5e27a19cd2d4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.364658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.365403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.365695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.366071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.370834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.371433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.371705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.371969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.373746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.375462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:43.389268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:43.390198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:43.391105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 381/756] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:43.391579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.443016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.444520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.460361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.545050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.553529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 376/757] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:57:43.553529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.582533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c8b0-354f-47d2-96fa-24426055931e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.647446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.648887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.649635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.650061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a97d8f-59e0-4408-9725-815d1d60f8ff None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.650678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.652523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.653321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.656230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.663604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.686744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 382/758] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.686744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.687112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.687112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.687112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.687112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.687607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.688223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.689499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.701797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 377/759] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.702790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.811842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780c81f-d46f-41ee-9089-7aafc0bba243 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.839714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87310f-1876-4d46-a36c-c6ca5312bfc9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.859342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.860626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.861635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.862172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.875850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.876553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.876669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.876938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.878290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=7c28473ae0554890a2b55941c15e94f4) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.878429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.878594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.880056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.880652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.894737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 378/760] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/role_assignments?user.id=7c28473ae0554890a2b55941c15e94f4&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:43.894737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.896567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:43.917909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:43.919004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:43.919920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:43.919920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:43.919920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:43.920953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 383/761] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:43.929786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.969325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.056864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3a24c-063b-42a9-bd6b-ccca35c4a4f5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.063710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0bcde-a323-4b10-a6fd-b8987760db0b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.098603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.099597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.100080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.100552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.101392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:44.101909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.103510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066014d0-a6e9-4e78-a44b-d0016df31d50 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.116525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.117092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 384/762] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:44.121655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.121655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.121655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.122931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=89d8460000674bcead45625785d81867, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.123682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b632-1aa8-4dea-a806-c97edd265b17 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.124176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b632-1aa8-4dea-a806-c97edd265b17 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.124348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d38de4-f650-46e8-a83d-67074708d813 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.124818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b632-1aa8-4dea-a806-c97edd265b17 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.126503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 385/763] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:44.127502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9b632-1aa8-4dea-a806-c97edd265b17 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.190610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.235421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 379/764] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:43 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:44.242786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.242786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.242786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.258492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9b632-1aa8-4dea-a806-c97edd265b17 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.274053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.275556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.337809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.338734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.339162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.339649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.351958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.355859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:44.380689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:44.381859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:44.383002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:44.391634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 386/765] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:44.399057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.427232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 380/766] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:44.472711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.508527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.560721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ead0-1212-428d-8a8b-de0992f3c0a5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.599336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a53d47-4f8c-4a06-8c2d-ef3000fc9379 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.603846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.640342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.641326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.641806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.642314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.643280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.643657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.644516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.644791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.645061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.645748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:44.646052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.646376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.651569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:44.664095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:44.665133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:44.665981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:44.669792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 381/767] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:44.674022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 387/768] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:44.675919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.681513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.806473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb681c8-df2b-4df6-aa60-73673d076762 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.818971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.843433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.844518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.845027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.845511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.846911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:44.847421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.848989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.854736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 382/769] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:44.860571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.913624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06b8e-b6ec-4cae-a622-48c309be263f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.954633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.956459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.956946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.957394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.958910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=2c61ea9321f94455afe35b931b9fcf8a) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:44.959409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.961519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102594-e454-471c-aa0e-0b6b39d8f0ff None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.969058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 388/770] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/role_assignments?user.id=2c61ea9321f94455afe35b931b9fcf8a&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:44.970483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e6a3-f382-430a-8546-0f448ee5ca17 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.970999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e6a3-f382-430a-8546-0f448ee5ca17 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.972035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e6a3-f382-430a-8546-0f448ee5ca17 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.973872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 389/771] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:44.986853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59fd9e-e033-47eb-af03-7e06fc7b6dad None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.989364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5d10-381e-469d-b932-51995128a46e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.989858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5d10-381e-469d-b932-51995128a46e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.990310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5d10-381e-469d-b932-51995128a46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.038745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-57dd5d10-381e-469d-b932-51995128a46e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.040013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-57dd5d10-381e-469d-b932-51995128a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.044633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.045545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.046025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.046476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.047365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:45.048865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.050641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.058395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 383/772] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.068956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.113481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.122159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.181165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 390/773] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:57:45.220169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5d10-381e-469d-b932-51995128a46e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.229368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2285a-c169-4c2c-8354-02a407421ba7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.275413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.275972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.275972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.276258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.276962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] RBAC: Adding query filter params (interface=public, service_id=185a73d7dce0482c88e6d1e6657957b2, region_id=RegionOne) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:45.277201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.278655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.290860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 384/774] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/endpoints?service_id=185a73d7dce0482c88e6d1e6657957b2&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.292368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.414552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5d10-381e-469d-b932-51995128a46e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.433665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.446609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fee492-2a70-4055-8fc5-444b2c9a32f8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.486652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.488116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.488392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.488659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] PATH_INFO: `/v3/services/network` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.489239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.490601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:45.506099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:45.507148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:45.507973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:45.508712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 391/775] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/services/network => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:45.519308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.533022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.533791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.534581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.534889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.535551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.536985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.546857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 385/776] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.583670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.641675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20890261-bc83-4d8a-a69d-1c1e0b5e3d0e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.683227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.683777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.684055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.684328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.684991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] RBAC: Adding query filter params (name=network) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:45.685297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.686816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.693113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 392/777] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.699053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.818289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f25e-35a6-472a-9b8a-db187b197fbd None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.853459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.854054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.854370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.854633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.855290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] RBAC: Adding query filter params (type=network) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:45.855583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.856937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.862137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 386/778] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.867816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.976538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.979465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e623c-af8a-4704-88e0-d6346cae3476 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.015576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.016187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.016471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.016748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.017454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] RBAC: Adding query filter params (interface=public, service_id=e1201dcb170c49e1830d88e3e331d9d1, region_id=RegionOne) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:46.017763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.019789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879800a6-29ef-4bb0-9ce5-e4fe00d0c2dc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.028463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 393/779] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/endpoints?service_id=e1201dcb170c49e1830d88e3e331d9d1&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:46.033993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.047120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:46.171087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a5077-3c52-469c-9d99-ddf5f42c6c3a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.207876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.208486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.208762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.209030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.209635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.211145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aa460-1b23-4c98-9051-cc6ef63da313 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.216818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 387/780] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:46.314488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b62b3a-f940-434b-bf52-474115160c3a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.315479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b62b3a-f940-434b-bf52-474115160c3a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.315913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b62b3a-f940-434b-bf52-474115160c3a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.317741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 394/781] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:46.323889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.324223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.324445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.351710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:46.352930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:46.447040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 388/782] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:46.470270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b62b3a-f940-434b-bf52-474115160c3a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.586996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b62b3a-f940-434b-bf52-474115160c3a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.630042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.630688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.631016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.631405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.648918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:46.654082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:46.655023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:46.655856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 395/783] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:46.661488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.664732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:46.798493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2f53d-39f4-4fdc-9113-04a8012eb268 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.845288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.845816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.846180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.846452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.847152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:46.847453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.848825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de726-53a4-4482-9af1-158860d09f28 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.862881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 389/784] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:46.868618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.881017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754867-df69-4dc4-b60f-06af853f942b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.881282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754867-df69-4dc4-b60f-06af853f942b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.881557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754867-df69-4dc4-b60f-06af853f942b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.886940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 390/785] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:46.888044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.889361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.889646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.927068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:46.930540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:46.931778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:47.023065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.028508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 391/786] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:47.050429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d985a5-3c2b-476b-93bd-ba5329c8b2fa None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.070600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.096765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.097638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.098221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.098772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.110849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.113038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:47.129722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:47.131005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:47.132116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:47.132116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:47.132116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:47.134056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 396/787] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:47.140165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.233751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.259477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1cc2f-6ff6-4209-b145-ff582ccbb8bc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.310515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.311612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.312103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.312570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.313385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:47.313902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.316288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.331926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 397/788] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:47.337730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.349439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c835ec3-e501-41ae-ba3d-df1b7002800d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.445828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111dc03-2f41-4254-aada-457cb7c6ad68 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.488039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.488938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.489365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.489781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.497502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.499273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.502959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.507421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.508069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.508346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.508615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:47.513134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:47.523835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 398/789] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:47.524795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.555350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-f200ee09-18ef-46aa-8119-10c6ba88bbd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:47.560283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:47.561428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:47.562562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 392/790] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:47.567555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16f84-0cde-4f70-8803-dd2933c8c917 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.567555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16f84-0cde-4f70-8803-dd2933c8c917 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.567555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16f84-0cde-4f70-8803-dd2933c8c917 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.570769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 393/791] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:47.571821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe16f84-0cde-4f70-8803-dd2933c8c917 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.647400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.648084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4720f5-75b8-4fcd-950f-e8be202959e1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.698781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.700476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.700962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.701681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.702952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:47.703490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.706819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5959e-2338-4f94-8de9-28a0dc6babaf None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.708881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe16f84-0cde-4f70-8803-dd2933c8c917 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.734775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 399/792] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:47.736487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.738137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.738553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.755513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.756478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.756975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.757544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.758184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:47.758621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:47.759143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.759381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:47.760935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.772830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 394/793] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:47.774007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.828615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.851607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 400/794] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:47.852597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.889756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b9af4-e67a-4abf-b857-db4526e5f2ff None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:47.910375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.929413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:47.930364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.930920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.931675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.943754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:47.945536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:47.964261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:47.965224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:47.966109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:47.966109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:47.966109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:47.968258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 395/795] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:47.969229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:47.987136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540933a-2315-4d1f-9d2f-66aedfb4b95b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.029652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.030279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.030556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.031270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.042918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.045002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:48.061181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.062623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:48.063557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:48.063557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:48.063557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:48.063557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 401/796] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.063557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.085547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98e95c-c727-4221-8241-32551bad4827 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.103675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.122011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.122636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.123040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.124026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.134492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-f248536a-303d-415b-a337-86fab9f5a87e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:48.139666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:48.141398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 396/797] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.142120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dde5-f9c3-47f4-a4ff-436712313d21 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.142193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dde5-f9c3-47f4-a4ff-436712313d21 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.143377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dde5-f9c3-47f4-a4ff-436712313d21 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.144723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 397/798] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:48.148872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815dde5-f9c3-47f4-a4ff-436712313d21 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.178722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac6390-7a19-4162-9ac8-58c1794b2739 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.181928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.215619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.216185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.216447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.216733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.217376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:48.217854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.219637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.238399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 402/799] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:48.239296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.256039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815dde5-f9c3-47f4-a4ff-436712313d21 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.295451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.296052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.296329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.296598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.297230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:48.297526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.298933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1298-34f1-4e60-8cc5-86d1f278cf9f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.311338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 398/800] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:48.312582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.312864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.313129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.332311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:48.333507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:48.346848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.353308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8e26-4edc-437e-912d-fc6e6dee3bb1 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.394759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.395370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.395640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.395925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.396568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:48.396866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.398232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.404220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 399/801] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:48.405272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.417580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 403/802] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:48.418650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.436862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.528995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f45f1c-75b8-4ec5-923a-35ecb42df21a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.531901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ac6c-9097-4e96-9250-25a8f702e787 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.569418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.570087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.570363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.570628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.574642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.575288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.575563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.575827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.579159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.581138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.583486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.585399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:48.592629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.593562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:48.594504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 404/803] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:48.598669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.604935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:48.607650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:48.608178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.608268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 400/804] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.608268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.626409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.686338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.705054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979033-10d0-414f-ab8f-18683d602d60 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.710814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bc819-aefc-45d1-9b9c-a8aad3d336fe None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.739662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.740279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.740559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.740825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.741401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.742779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.743780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.744396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.744663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.744926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.751431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.752591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:48.753564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 401/805] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.754363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.761441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.762909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:48.777418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:48.778378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:48.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:48.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:48.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:48.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 405/806] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:48.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.848040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.856411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3452-ece9-41ef-abba-bac9f07e8d72 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.879816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca16a8a-1652-4f67-a29b-b8a43a648800 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:48.889506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.890102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.890368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.890631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.891394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:48.891686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.893991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe01dd-5d48-4826-8a96-324e9f2bc956 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.910609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 402/807] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:48.912047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685ffcc-d268-4a0c-a837-edb657dddc35 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.912361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685ffcc-d268-4a0c-a837-edb657dddc35 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.912590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685ffcc-d268-4a0c-a837-edb657dddc35 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.913916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 403/808] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:48.914277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:48.914497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685ffcc-d268-4a0c-a837-edb657dddc35 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:48.915132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:48.915312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:48.915585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:48.916334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:48.916639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:48.917563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:48.918060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:48.930781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 406/809] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:48.931587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.016548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685ffcc-d268-4a0c-a837-edb657dddc35 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.033004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902e2e-9b75-4735-b4f1-bfe4926aa68c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.051266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.051856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.052132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.052399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.053044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:49.053369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.054745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.059213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 404/810] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:49.059968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.061340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.068081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.068682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.068953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.069217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.078275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:49.078275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.078275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2446-971b-4def-9d2c-b3e9253d3c93 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.082906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 407/811] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:49.084143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.084428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.084703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.102445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:49.103674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:49.126348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.160638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c841ac-1acb-45e4-8a88-21911b6ecad2 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.170320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 408/812] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:49.171158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.201695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.202289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.202561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.202871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.268272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b97bf6-1874-4232-b075-15705657cbe0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.269180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.270625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.288571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.302475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.303153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.303423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.303690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/7c28473ae0554890a2b55941c15e94f4/roles/666454a6407846b4936d1df7c782f605` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.318601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=7c28473ae0554890a2b55941c15e94f4, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.320184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.333586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.367703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 405/813] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:57:49.368540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.378547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 409/814] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:49 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/7c28473ae0554890a2b55941c15e94f4/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:49.379739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.488244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2d64-1b26-452a-bc09-be1e28e8bd28 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.516050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf25acd-bb8f-4f4b-8706-f04c36b0de73 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.528344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.528886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.529155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.529425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.530664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:49.530664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.532441 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.540619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 406/815] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:49.541492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.561954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.562587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.562925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.563202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.572142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.573650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.577787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:49.586665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:49.587693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:49.588656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 410/816] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:49.589246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.623284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.658744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b5e8f-b6c0-4148-a7d6-29d1fd8dc4f5 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.694651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.695306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.695613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.695884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.696461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.697833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:49.707285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:49.708439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:49.709280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 407/817] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:49.710081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.718996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a396cc-9dcc-4300-9343-27f3431d3bf8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.757873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.758890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.759172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.759456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.771242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.772684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:49.787404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:49.788377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:49.794096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:49.794096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:49.794096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:49.794096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 411/818] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:49.794096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.814287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.821698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd3910-abd8-4661-a466-01cedf4804b0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.858642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.859272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.859559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.859814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.860467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.862125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.882066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:49.901198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1bee7-24f2-4b9d-a026-58f5a9b44403 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:49.934877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:49.935481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:49.935762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:49.936031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:49.936747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:49.937048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:49.938538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:49.956973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 408/819] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:49.958062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:49.970533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 412/820] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:49.970533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.103009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526bc12-4862-45d7-b154-eef1a703bc38 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.110535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea12-2f6c-4b3d-a63d-e48a49ef78d5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.151155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.151760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.152033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.153323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.153323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.153323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.154644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.155546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.156431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.156715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.157479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.158123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] RBAC: Adding query filter params (name=network) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.160779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.160779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.165183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.166620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 409/821] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:50.168583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.170686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 413/822] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:50.176216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.191091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.296835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc70df-c76c-4e2d-9e95-06f4f6677c32 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.310806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff684-a6ce-4cf2-b3bd-29bcb26a042e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.341088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.342209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.343079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.343770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.347985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.348612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.348889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.349157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.352996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.354647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.360696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.362238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:50.369505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:50.370565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:50.371683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:50.379419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 414/823] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:50.379576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:50.380622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:50.381723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:50.381723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:50.381723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:50.382977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.387874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 410/824] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:50.388894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.465108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.467369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.521025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90fee-8afe-4641-ae79-03854b437e73 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.525855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154eef-0249-4dae-8b00-b88325dfd575 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.560463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.561334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.561447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.561697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.562720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.562923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.564494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.571022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.571921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.572419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.572884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.574284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] RBAC: Adding query filter params (type=network) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.574818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.577228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.581910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 411/825] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:50.581910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.583546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 415/826] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:50.585761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.704628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5db0-895e-4240-a6eb-e22635c9e485 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.708842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60d28e-0b0b-4c3b-9119-40017ef9313e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.730166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.741886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:50.743914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.744660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.744946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.745220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.746003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.746146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.747537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.755311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.763034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.763034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.763034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.763034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:50.763034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.763400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 412/827] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:50.763476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.763564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.768540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 416/828] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:50.770207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:50.886935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb38fd3-8444-4bb9-9f22-1eea8a63e23b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.907979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849199e-fb64-4fc1-bfc0-cc58fb8bee99 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:50.944270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.945308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.947165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.947657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.948547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.951263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.966621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:50.967928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:50.969971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:50.970462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:50.983874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:50.985740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:50.999597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:51.000840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:51.001849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:51.005060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 417/829] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:51.010309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:51.047694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 413/830] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:50 2022] POST /identity/v3/endpoints => generated 337 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:51.073980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:51.144041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca09fe3-5bcd-4264-a757-14a97bbde0dc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:51.188466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:51.189081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.189362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.190173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.191003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:51.191323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:51.192918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcbf6b-c7ae-4a69-9361-3d6d4c2de71a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:51.207929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 418/831] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:51.214148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:51.266289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:51.338169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488adb-5946-44da-9218-bd1ba0ab7047 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:51.379094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:51.379751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.380024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.380292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/2c61ea9321f94455afe35b931b9fcf8a/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.383393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=2c61ea9321f94455afe35b931b9fcf8a, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:51.384824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820cea-b408-4175-8db3-fe32e200e6bb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:51.443921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 414/832] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:57:51 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/2c61ea9321f94455afe35b931b9fcf8a/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:51.500033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c680b-9ac5-40d0-b301-d9576c1a3792 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.500451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c680b-9ac5-40d0-b301-d9576c1a3792 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.500655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c680b-9ac5-40d0-b301-d9576c1a3792 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.502168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 419/833] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:51.508686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.509288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.509756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.530866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:51.532450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:51.612271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 415/834] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:51.638334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c680b-9ac5-40d0-b301-d9576c1a3792 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:51.758931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c680b-9ac5-40d0-b301-d9576c1a3792 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:51.761616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:51.800150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:51.801163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.801637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.802100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.813285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:51.819065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:51.819989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:57:51.820830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:51.828393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 420/835] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 17:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:51.830637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:51.872276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:51.949293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e368b12-07b3-4029-95de-3949c015b627 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:51.984463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:51.985063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:51.985334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:51.985600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:51.986904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:51.987210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:51.988682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:51.997093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 416/836] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 17:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:52.003064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.113498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a23a4-bbbc-4e26-95e1-bfa837a5c451 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.151426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.152851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.153334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.153806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.165373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.167225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-40103a5a-a0af-4f1e-bfb4-95622402bbf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:52.182391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:52.183420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:52.184315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:52.184315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:52.184315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:52.190806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 421/837] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:52.193040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.257759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc499-28cc-4be2-ab7e-541a748640d7 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.258618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc499-28cc-4be2-ab7e-541a748640d7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.259131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc499-28cc-4be2-ab7e-541a748640d7 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.261082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 422/838] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:52.271507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.271819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.271999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.290521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:52.291787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:52.303848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.309886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.323148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22389c-9ef0-47db-9e76-5474d753379e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.359525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.360129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.360403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.360670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.361893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:52.362377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.364040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.380015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 417/839] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:52.380203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 423/840] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:52.383580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.409561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.523595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddde4-eda8-4574-965b-1c973f795a85 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.535515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef887310-5a65-4286-9f90-801cf7f64468 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.576067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.577128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.577852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.577971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.578398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.578680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.579043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.579382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.580043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.581616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.587924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.589614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:52.592153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:52.593414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:52.594676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:52.601531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 424/841] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:52.603793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:52.610653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:52.612512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:52.613695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:52.619420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 418/842] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:52.623630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.626724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.639066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.719346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2ae6-332c-4b21-ba41-b1bd2d66504b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.741129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a05b2-833e-402a-8c98-2282cc2b3f31 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.765971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.766672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.767049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.767987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.769037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:52.769529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.771365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.778738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 425/843] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:52.785041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.788362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.788935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.789211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.789480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.790734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:52.790734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.792082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.815093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 419/844] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:52.817119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.888005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.895588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:52.924798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d407ad6-f7dc-4973-969e-b51149c4af91 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.931523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d302ff-6908-49c8-b793-dd956c71ad67 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.969473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.970433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.971075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.971582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.972415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:52.972940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.973640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.974248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.974519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.974805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.975451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.987230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 426/845] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:53.056282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.057747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caf907-82c6-4c27-be5b-751c0958204a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.132556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.171317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 420/846] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:57:53.177595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.227694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c8ba-35f9-41cb-8441-1b54ae34021e None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.228336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c8ba-35f9-41cb-8441-1b54ae34021e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.228805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c8ba-35f9-41cb-8441-1b54ae34021e None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.230570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 421/847] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:53.240287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.241020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.241622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.261265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:53.263059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:53.286597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea662-5cbd-4447-a4c8-7a6802b934fe None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.327105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.328107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.328585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.329051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.341128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.344925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.347074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.354837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 422/848] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:53.363752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:53.364684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:53.365657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:53.365657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:53.365657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:53.365657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 427/849] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:53.370671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.374509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.492042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad90d4-4c0e-4598-ba14-e7879fea1f92 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.506495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.511003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e9eb-ce1e-4c9e-a2bc-802f34f2329f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.528786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.529806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.530295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.530804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.531600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.533907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1dba45b8-1bc6-46ff-91fe-43d16d0bc24a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:53.543818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:53.544977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:53.546061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:53.547318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 428/850] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:53.548640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ae15-22b3-4d3a-8be5-b656b5ac9721 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.549881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ae15-22b3-4d3a-8be5-b656b5ac9721 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.550404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ae15-22b3-4d3a-8be5-b656b5ac9721 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.555967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 429/851] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:53.555967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ae15-22b3-4d3a-8be5-b656b5ac9721 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.561700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.561700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.561700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.561700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.561983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:53.561983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.562325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ed51-14e1-487e-b95a-6ac3bb78cca8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.575837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 423/852] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:53.577345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.577654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.577964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.596960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:53.598257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:53.617972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.666344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ae15-22b3-4d3a-8be5-b656b5ac9721 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.678729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 424/853] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:53.678729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.708840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.710003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.710475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.710970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.711924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:53.713165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.714751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.722956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 430/854] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:53.723751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.772382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.785761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06cfe2-3b90-425b-9dab-4dbf07ef5ecc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.826873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.827499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.827777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.827993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290003e8-75b9-4e50-a0ef-c24751140144 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.828095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.836699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.838970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.845029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:53.853605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:53.861440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 425/855] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:53.862453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.864046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.864674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.864949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.865213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.877039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:53.883229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:53.884154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:53.884984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 431/856] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:53.885680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.968995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95cd53-a36e-4016-8da6-164e286cfc25 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.994867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387802-b375-46f6-bab6-ef1c2c928e8e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.007056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.007883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.008315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.008728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.009508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.009807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.011201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.015739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 426/857] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:54.024488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.035672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.037528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.038066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.038549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.040091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.041110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.043259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.044011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.063432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 432/858] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:54.070117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.130529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.148288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597833-08ba-4c80-8a1f-a709b395b998 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.187282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.188023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.188392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.188746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.190219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.190515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.192156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.203088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 427/859] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:54.207713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.218162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe89470-40ec-41c7-85cd-4509dbf0e814 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.261112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.261633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.262085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.262366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.263500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=7c28473ae0554890a2b55941c15e94f4) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.264082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.265707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.273733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 433/860] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/role_assignments?user.id=7c28473ae0554890a2b55941c15e94f4&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:54.299706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.325897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67356-47a3-41be-9b26-71dbaa62b02f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.365217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.365902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.366141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.366413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.379386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.381000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:54.397828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:54.398913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:54.400267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:54.400267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:54.400267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:54.400267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 428/861] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:54.404825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.513252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876c7b-e9cf-4deb-a27d-662cf3309e28 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.514739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.551125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.551795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.552076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.552345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.552985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.553282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.554652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.568111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 434/862] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:54.573711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.676535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a13647-1407-47b7-9849-52d8d1ed221f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.702386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.714577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.715237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.715513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.715782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.724446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.725904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:54.737873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:54.738883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:54.739837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 429/863] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:54.744663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.849189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a90ae0-5917-4b79-bc49-8a4c50fd854f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.883895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.884534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.884865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.885139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.885847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:54.886144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.887645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ed28e-1ade-482e-9b90-161d443b892c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.905777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 435/864] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:54.911991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.021546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ecddc-c8e3-458d-b478-50cf87673a88 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.055744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.056344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.056622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.056892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.095227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9aa82-6b17-4ec0-8aea-07a99b09f27b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.095665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9aa82-6b17-4ec0-8aea-07a99b09f27b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.095845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9aa82-6b17-4ec0-8aea-07a99b09f27b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.098816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 436/865] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:55.102549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.102816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.103094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.117872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.121173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:55.122372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.130311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.131998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.137706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.218835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 437/866] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:57:55.234212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.240491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 430/867] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:57:55.247360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.334273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.349552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb3d-892e-4747-ba35-1078b42f2a71 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.353810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8aa3e-ec16-4a31-a6ce-466eebd26fc2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.384241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.384935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.385120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.385458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.393790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.394808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.395321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.395788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.396628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.396826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.398241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.398596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:55.413234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:55.414233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:55.415166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:55.415166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:55.415166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:55.418987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 431/868] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:55.419681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.479056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 438/869] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:55.540416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fded2f-1494-4c47-bcb2-4eed338f6caa None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.590534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766894d1-3337-4583-8306-e08732234246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.591545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766894d1-3337-4583-8306-e08732234246 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.592030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766894d1-3337-4583-8306-e08732234246 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.592482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766894d1-3337-4583-8306-e08732234246 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.593332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766894d1-3337-4583-8306-e08732234246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:55.593813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766894d1-3337-4583-8306-e08732234246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.595946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766894d1-3337-4583-8306-e08732234246 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.597880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.612889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 432/870] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:55.618076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.647119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.729742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce6354-7c03-486d-976e-8d6c85550379 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.764799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.765485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.765762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.766029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.773173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.774611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:55.786885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:55.788912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:55.791020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 439/871] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:55.794208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766894d1-3337-4583-8306-e08732234246 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.897698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766894d1-3337-4583-8306-e08732234246 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.933952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.934902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.935349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.935784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.936651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:55.937119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.938698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597238-de84-4284-adae-a6e252af2ff9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.951092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 433/872] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:55.957480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.030877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.065319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833262ff-f82e-4bbe-ad34-fbb645678bd4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.101884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.102811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.103263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.103742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.104635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=2c61ea9321f94455afe35b931b9fcf8a) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:56.105111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.106778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3df4c-8a62-424b-b368-739ac1e20bef None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.113569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 440/873] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/role_assignments?user.id=2c61ea9321f94455afe35b931b9fcf8a&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:56.203483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e5cf0-cd05-4252-abe3-b946745f5aff None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.204090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e5cf0-cd05-4252-abe3-b946745f5aff None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.204552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e5cf0-cd05-4252-abe3-b946745f5aff None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.207258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 434/874] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:56.212625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8828d-2df8-4e4d-826d-a8f5b306fbae None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.212959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8828d-2df8-4e4d-826d-a8f5b306fbae None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.213164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8828d-2df8-4e4d-826d-a8f5b306fbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.233141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-eda8828d-2df8-4e4d-826d-a8f5b306fbae None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:56.234331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-eda8828d-2df8-4e4d-826d-a8f5b306fbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.308666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 441/875] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:56.330697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e5cf0-cd05-4252-abe3-b946745f5aff None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.391857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.444179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e5cf0-cd05-4252-abe3-b946745f5aff None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.490802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.491410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.491683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.491949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.492586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.494031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.513550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8bf8-4cc0-4697-9dbc-538831685970 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.518982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8bf8-4cc0-4697-9dbc-538831685970 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.518982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8bf8-4cc0-4697-9dbc-538831685970 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.518982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 442/876] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:57:56.521268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.521550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.521819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.541451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:56.542640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.604892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 435/877] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:57:56 2022] POST /identity/v3/services => generated 269 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:56.610591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.628852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 443/878] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:57:56.653473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.775299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ac208-1d54-4deb-b852-0fa871536a20 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.790943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.811924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.812615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.812889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.813158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.824968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:57:56.830342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:56.831447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:56.832288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 436/879] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:56.837905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.945341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322064-b851-4606-830b-faddfb79cbb5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.981136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.981738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.982012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.982429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.983136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:56.983437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.984799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.993600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 444/880] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:56.999405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.110437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a38f-8ea0-4fd9-beab-19d41f0b1d89 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.145640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.146259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.146536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.146836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.158500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.160195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:57.179499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:57.180497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:57.181376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:57.181376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:57.181376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:57.181376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 437/881] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:57.187791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.244886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.296067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d909c-c19b-4653-9ef9-314adc105e0e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.332216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.332844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.333112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.333384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.334036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:57.334334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.335758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.350145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 445/882] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:57.359808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.393094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.487194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326c99-78fc-4611-8460-3ce246c2a5f9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.522810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.523433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.523768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.524041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.531426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.533596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e08e018a-6a3b-40e8-90d1-d06767c4b7ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:57:57.545603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:57.546559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:57.547551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 438/883] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:57.555912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.666144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eab6-72ff-4ce4-9e2a-524517f3b4bd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.679871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d813f82-91ad-4666-80ec-f33a79affe36 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.680144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d813f82-91ad-4666-80ec-f33a79affe36 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.680418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d813f82-91ad-4666-80ec-f33a79affe36 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.680531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.683061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 439/884] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:57.687044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.687327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.687600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.705187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.705802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.706082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.706355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.706565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:57.707115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:57.707417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.707886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:57.708852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.733099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 446/885] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:57.739055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.800032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 440/886] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:57:57.819138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.841153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.852228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd766db-7da6-45c7-913c-6e7e8a6159d9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.894572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.895311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.895657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.896002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.941110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880807-267b-4a64-a05c-7aff538c550f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.951103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.983014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.984512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.989806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.990511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.990913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.991255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.991954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.993418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:58.006045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:58.007138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:58.008008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 441/887] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:58.013611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.107879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.109535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 447/888] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:57:58.115917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.134220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc82da-b102-4494-a31b-c7beb3a0a17b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.172109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.173169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.173676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.174180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.175075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:58.175590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.177239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.183067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 442/889] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:58.189886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.201441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.239910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641ec6-0ed3-40ab-a1f6-13e0a62d6f99 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.278140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.278822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.279078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.279349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.290437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.291916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-cd41e8e8-251c-415b-90ee-484396977f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:57:58.306944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:58.307907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:57:58.308783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:57:58.308783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:57:58.308783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:58.308783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 448/890] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:57:58.313748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d567-8f41-4593-9020-b4df6c71466f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.314072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d567-8f41-4593-9020-b4df6c71466f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.314341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869d567-8f41-4593-9020-b4df6c71466f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.315619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 449/891] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:58.316302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869d567-8f41-4593-9020-b4df6c71466f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.321973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3513-6b92-47dc-82b1-8d8a8aa0947c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.357236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.357835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.358111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.358379 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.359036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:58.359332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.360700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1718c02-9ce6-4ce6-91fe-9e72664a22ff None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.365714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 443/892] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:58.366865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.367147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.367414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.388548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:58.390359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:58.392896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.435496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869d567-8f41-4593-9020-b4df6c71466f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.460808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.479767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 444/893] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:58.479767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.482892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.483624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.483980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.484352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.485059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:58.485468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.488269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.503582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 450/894] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:58.504337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.590024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff76ff-143e-4547-9b1d-5d5492179260 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.608402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9768c-457c-44f2-9a13-95db0762c1a0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.626297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.626957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.627228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.627495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.628183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] RBAC: Adding query filter params (interface=public, service_id=d99dbe52cd7e44b6bd0341e32ae0f20a, region_id=RegionOne) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:58.628484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.629846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.634776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 445/895] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/endpoints?service_id=d99dbe52cd7e44b6bd0341e32ae0f20a&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:58.635545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.638047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.650598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.651277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.651556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.651820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.652454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.653735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:58.663371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:58.664389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:57:58.665213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 451/896] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:58.665996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.713300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.754808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c0c9d-15c8-4d1c-8025-ecef065f4fd9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.771504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be347d-d5ab-485c-a357-31d5a68b0129 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.793809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.794492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.794855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.795129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.803162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.804726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.810103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.810874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.811164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.811437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.812070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.813426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:57:58.817374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:58.818460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:57:58.819588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:58.820258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 446/897] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:58.821085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.826912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 452/898] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:58.844298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.899253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.942539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d5e2-e78a-4dd1-9111-48de2b375acb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.962022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb98fb-f4cf-4d00-ace6-3d5e0150e8a8 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.974488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.981638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.981954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.982347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.982620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.983280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:58.984090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.984947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9087690-ed16-4697-a889-887c757f69a5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.990843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 447/899] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:58.992026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca65e2-7141-4b21-8eab-080cfceb41dd None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.996336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca65e2-7141-4b21-8eab-080cfceb41dd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.996336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca65e2-7141-4b21-8eab-080cfceb41dd None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.996336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 448/900] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:57:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:58.998429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca65e2-7141-4b21-8eab-080cfceb41dd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.011030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.011955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.012231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.012503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.013206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.013502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.014967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80502-c026-4446-8cd1-9c2e1ac721c5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.027962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 453/901] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:59.029336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.029620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.029895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.048368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:59.049754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.110246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca65e2-7141-4b21-8eab-080cfceb41dd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.130859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 454/902] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:57:59.131765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.136852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:59.146568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.147307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.147592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.147865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.148493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.149948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.150260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.155154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 449/903] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:59.164057 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.265730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:59.270773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6c9cd-fc70-458f-b8bb-4cddeb2c865c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.287991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d4da4-2e88-4a19-8c30-3b8c2bfb7887 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.313064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.314639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.315132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.315659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.316586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] RBAC: Adding query filter params (role.id=4ef7dec740ba4329ae1e895d9d41cc91, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=7c28473ae0554890a2b55941c15e94f4) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.317105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.318794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.325928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 455/904] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/role_assignments?user.id=7c28473ae0554890a2b55941c15e94f4&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=4ef7dec740ba4329ae1e895d9d41cc91 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:59.326236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.327328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.327328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.327522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.327991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.329371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:57:59.339856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:57:59.340864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:57:59.341768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 450/905] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:57:59.348704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.504993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8ba4c-4107-4420-b51c-50a7ab6c501a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.565588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.566652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.567238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.567772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.568909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.569497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.571938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.580990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 456/906] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:59.589736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.690077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:59.767411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915d6da-f616-45f6-860f-a5e47509c086 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.825203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.826102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.826576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.827072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.828227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.828806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.831954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.841024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 451/907] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:59.850806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.899958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:00.027655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24151c08-e40a-410d-a64d-c73b78fe2fe0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.071306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.071911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.072185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.072519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.073139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] RBAC: Adding query filter params (interface=public, service_id=55f9b0a491c54250a3c794d21cb977fd, region_id=RegionOne) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:00.073442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.074816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578468ff-4062-4d3e-a67f-2138c4a9df07 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.081451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 457/908] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/endpoints?service_id=55f9b0a491c54250a3c794d21cb977fd&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:00.086812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.191285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743e079-2dc3-4afe-a538-dad48d9a28c0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.208778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:00.225590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.226188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.230019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.230019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.230019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.230019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201a731-d60e-40a7-b092-fb7719faa2cf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.234933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 452/909] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:58:00 2022] GET /identity/v3/services => generated 1268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:00.945502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4db9c-e274-412a-9b28-5b02efb1c8cd None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.946029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4db9c-e274-412a-9b28-5b02efb1c8cd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.946029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4db9c-e274-412a-9b28-5b02efb1c8cd None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.947757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 458/910] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:00.953101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.953501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.953666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.973622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:00.974815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.043622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 453/911] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:01.066878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4db9c-e274-412a-9b28-5b02efb1c8cd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.166055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.176549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4db9c-e274-412a-9b28-5b02efb1c8cd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.211144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.211692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.212177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.212603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.213636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.215121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-216be606-765d-4653-a7ce-d41f82c3d768 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:01.224788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:01.225829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:01.226666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 459/912] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:01.232901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.261244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782817-5fd9-4d71-9a5c-8d89484724d2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.261629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782817-5fd9-4d71-9a5c-8d89484724d2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.261820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782817-5fd9-4d71-9a5c-8d89484724d2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.263441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 460/913] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:01.269296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.269573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.269850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.292686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:01.294651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.344744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bddc9-b457-4b7b-8d32-14a4fe87ec54 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.359575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.380767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.381388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.381661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.381926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.382692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:01.382912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.384331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.389972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 454/914] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:01.398892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 461/915] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:01.400222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.426564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.532664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.546792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ebf00-bb3e-4305-8f2f-f56b72f94674 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.556113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c87fa8-298d-428e-a7bc-dfa8a131744c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.585370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.585979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.586258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.586529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.587186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:01.587497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.588856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.599832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 455/916] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:01.601160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.602681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.603878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.604349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.605759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.606622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.608202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8f66c-70fc-4002-8f57-688fa9ca147d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.645746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.688245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 462/917] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:58:01 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:01.689954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06641bd5-96bb-4b8f-b357-aab2a9ef74fd None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.690239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06641bd5-96bb-4b8f-b357-aab2a9ef74fd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.690508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06641bd5-96bb-4b8f-b357-aab2a9ef74fd None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.692232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 463/918] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:01.704515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.704852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.705074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.708929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81173326-5061-49a2-8c38-98dab6516860 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.735369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:01.737985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.758091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.758773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.759052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.759315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.759967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.761693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.830932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 464/919] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:01.838979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 456/920] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:01.861833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.956943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.984557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6acb37-c616-4c64-a74a-cec3e84a7a2a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.031835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.032417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.033283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.033557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.044154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:02.048928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:02.049881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:02.050787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 465/921] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:02.058248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.156504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.166957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853174a8-c5b7-472a-9221-3d533e8c56ce None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.201101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.201708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.201987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.202256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.203004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:02.203311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.205283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a6dcd-2758-4844-a9c3-6cd8d6a5bf50 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.214466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 457/922] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:02.221642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.322109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c40552-8a94-4bb0-818e-b9a068b2d940 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.322430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c40552-8a94-4bb0-818e-b9a068b2d940 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.322616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c40552-8a94-4bb0-818e-b9a068b2d940 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.325444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 458/923] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:02.331439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.331707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3b813-77fd-4efe-baf2-9fb35aa26312 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.331814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.331971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.351046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:02.352239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:02.364351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.369634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.370261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.370535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.370830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.382251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.383836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:02.402304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:02.403434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:02.404723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:02.404723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:02.404723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:02.404723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 466/924] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:02.409465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.447560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 459/925] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:02.473419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.524028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc2f43-2008-4697-a179-7822c956393f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.561092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.562024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.567294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.567294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.567294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:02.567294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.567294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.580347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 467/926] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:02.585462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.592668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.613395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def9159-e019-46cf-8fc1-3c73c30241e0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.643536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.661017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.662079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.662490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.662789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.663416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.664828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:02.675788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:02.676720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:58:02.677639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:02.685200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 460/927] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:02.686288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.711584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6961c9-42de-4f0f-b841-9384a2f06276 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.759386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.760013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.760296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.760564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.773098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.774853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:02.787690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:02.788654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:58:02.789582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:02.796525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 468/928] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:02.797551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.797921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f4230-e7d9-4ef3-a818-21d80dcb550c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.844375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.845005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.845736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.845989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.846610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:02.846932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.849070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.857600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 461/929] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:02.860688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:02.884093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.920977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fddb44-3f93-465e-9f86-4c73c8f321f7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.924262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:02.966214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:02.966917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:02.967000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:02.967212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:02.967944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:02.968239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:02.969665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:02.988206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687860ac-c635-4804-97bd-9c27f248b0b6 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:02.990006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 469/930] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:02.999831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.030389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.031862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.032467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.032931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.033993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:03.034906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.036775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.043589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 462/931] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:03.051527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.123279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0011f4-76cf-41f5-8245-fa091eba434e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.141239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:03.163336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.164224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.164659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.165074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.170289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7533-58c8-489a-b065-d3295f90f2ca None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.177742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.179502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:03.195904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:03.196878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:03.197771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:03.197771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:03.197771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:03.205092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 470/932] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:03.206244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.209997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.211448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.211724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.211992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.212644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.214006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.224729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:03.314494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43eba-0d4b-42d2-8b11-2cf7288a0da6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.330208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 463/933] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:03.364378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.365246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.365668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.366125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.367014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:03.367487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.369132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.383264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 471/934] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:03.388960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.518395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8b1f-3de6-419a-926c-16a744e678a3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.549118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:03.556177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.556808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.557084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.558171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.564421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.566046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:03.578593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:03.580573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:03.581765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 464/935] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:03.589312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.663891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:03.698490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623af5-63e4-4f45-8e4e-d24216bb7c56 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.735157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.736263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.736637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.737010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.737792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:03.738194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.739737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657da53-21af-4744-b04d-6bbee81a1db1 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.752916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 472/936] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:03.759202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:03.866230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f274e-9f02-43b2-8947-3545bb0b3a00 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:03.894548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdef620-873f-45e1-a2ca-dc7fbaa90047 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.894869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdef620-873f-45e1-a2ca-dc7fbaa90047 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.895143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdef620-873f-45e1-a2ca-dc7fbaa90047 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.896726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 473/937] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:03.902557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.902868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.903146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.905397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:03.906423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:03.906891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:03.907444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/7c28473ae0554890a2b55941c15e94f4/roles/4ef7dec740ba4329ae1e895d9d41cc91` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:03.910387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=7c28473ae0554890a2b55941c15e94f4, role_id=4ef7dec740ba4329ae1e895d9d41cc91)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:03.913220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d210d-7dc1-4aa9-9f3c-b9f3d6652f9d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:03.926324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:03.927729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:04.005334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 474/938] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:04.020466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 465/939] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:58:03 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/7c28473ae0554890a2b55941c15e94f4/roles/4ef7dec740ba4329ae1e895d9d41cc91 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:04.030212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.043047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ce3d-9801-41b3-8b64-2b95a3603915 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.043332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ce3d-9801-41b3-8b64-2b95a3603915 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.043610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ce3d-9801-41b3-8b64-2b95a3603915 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.045022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 466/940] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:04.051147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.051428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.051709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.070762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:04.071991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:04.115915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.127698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.157935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 467/941] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:04.162628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc14ef-d514-4a1c-814f-8a128a889943 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.181434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.206820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.207725 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.207981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.208343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.209275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.210512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:04.220016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:04.221165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:04.222082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:04.226409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 475/942] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:04.228016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.320770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d36c3f-01ed-4d85-aee3-d2aa5bd9d8c8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.343974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10adc7-1444-4188-b27a-05fca265d4d6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.369930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.370483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.370790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.371063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.371766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.373220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.374693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.375885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING keystone.server.flask.common [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 17:58:04.387760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.390510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.390510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.390510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.390510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:04.390510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.392559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.397668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 476/943] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:04.403318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.435519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.sql.core [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:58:04.435519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 17:58:04.435519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [parameters: {'id': '8c826c00cb834faaa46414a45db8134e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 17:58:04.435519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80545) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c826c00cb834faaa46414a45db8134e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c826c00cb834faaa46414a45db8134e. Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:58:04.461427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result.read() Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 17:58:04.462381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application yield resource Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise self.value Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application session.commit() Aug 09 17:58:04.463393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:58:04.464303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:58:04.465200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result.read() Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:58:04.465996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [parameters: {'id': '8c826c00cb834faaa46414a45db8134e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.468257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:04.470621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c826c00cb834faaa46414a45db8134e. Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 468/944] 10.222.0.36 () {60 vars in 1140 bytes} [Tue Aug 9 17:58:04 2022] POST /identity/v3/projects => generated 262 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:04.473102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.521903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eefc9-c112-4ffd-b6d0-8e8acd0670a9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.534525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.569598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.570609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.571007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.571377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.572168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:04.572567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.574912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.582871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 477/945] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:04.588912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.624059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283010b9-19b9-42de-b8ec-d76c8b5ea0cf None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.650379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.678027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.678691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.678990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.679257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.690466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.692738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.701297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4c8a0-cf19-4269-bc08-409e05affca6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:04.707989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:04.709068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:04.710150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 469/946] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:04.714872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.744763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.745457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.745804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.746196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.746993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] RBAC: Adding query filter params (interface=public, service_id=9bb252b581034051a5f72e1dd6e3c84a, region_id=RegionOne) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:04.747366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.748875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.758934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 478/947] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/endpoints?service_id=9bb252b581034051a5f72e1dd6e3c84a&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:04.760792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:04.854010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.860454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be428438-4d8b-4327-9645-c4e2e82b0be3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.874244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d79e38-aa7b-4660-8ed3-237dfe82613e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:04.898813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.899317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:04.899647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.900040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.900391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.901637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:04.901915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.903510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca6791-593e-40a6-85b0-20e0211058c5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.921357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:04.922189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:04.922617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:04.923173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:04.924024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:04.929188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 470/948] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:04.929380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143b68-103f-4fd6-b772-b261aa8c984f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:04.936360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 479/949] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:58:04 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:05.575849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79112-e2dc-4388-a1ea-fc784ffbc14a None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:05.576371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79112-e2dc-4388-a1ea-fc784ffbc14a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:05.576371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79112-e2dc-4388-a1ea-fc784ffbc14a None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:05.578084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 471/950] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:05.583106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:05.583506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:05.583661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:05.602996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:05.604774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:05.675962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 480/951] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:05.697468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c79112-e2dc-4388-a1ea-fc784ffbc14a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:05.809299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c79112-e2dc-4388-a1ea-fc784ffbc14a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:05.845508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:05.846398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:05.846900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:05.847358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:05.848106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:05.849635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:05.858639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:05.859849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:05.860947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:05.863355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 472/952] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:05.871836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:05.971459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655eca-5167-4523-b2f3-1d9ed83fc2b6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:05.982115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:06.005686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.006294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.006569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.006868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.007494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:06.007790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:06.009154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2834e7-ac7b-49c9-8926-60b525b16fc9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:06.014477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 481/953] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:06.020998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:06.047977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:06.131337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f349-8198-431c-8c07-cc9eeca79c4d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:06.169006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.169638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.169912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.170179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.170865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] RBAC: Adding query filter params (type=compute) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:06.171164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:06.172619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07755d1-c2fb-470b-b3bd-a4d5d5fbcfda None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:06.178877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 473/954] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:06.197545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc3aa-a943-4484-a61a-a3814d921eb2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.197923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc3aa-a943-4484-a61a-a3814d921eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.198266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc3aa-a943-4484-a61a-a3814d921eb2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.200090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 482/955] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:06.206886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.207272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.207617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.227761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:06.229425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:06.314851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 474/956] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:06.332101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fc3aa-a943-4484-a61a-a3814d921eb2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:06.437803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fc3aa-a943-4484-a61a-a3814d921eb2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:06.472674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.473304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.473632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.473900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.484580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:06.490160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:06.492276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:06.494218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:06.498788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 483/957] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:58:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:06.498968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:06.517317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:06.604140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f3b07-be87-4d0f-adfe-9647cfa16959 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:06.640627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.641228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.642098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.642501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.643513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:06.643812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:06.645179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:06.653961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 475/958] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:06.660644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:06.699650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:06.770574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ffe3f-9042-4357-abbf-3128d07eafa6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:06.807714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.808346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.808623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.808894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.819832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:06.821291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:06.836646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:06.838800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:06.840834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:06.840834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:06.840834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:06.840834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 484/959] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:06.843753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:06.930511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:06.952770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaed8-f99c-4afc-a59c-672977bf1b3d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:06.988408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:06.989005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:06.989281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:06.989547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:06.990181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:06.990477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:06.991932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09209f8-573c-458b-9d87-5224679fcbe3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.004684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 476/960] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:07.010040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.100903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.116706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c38c6-21d8-444e-9dd8-67839d2ef803 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.150994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.151620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.151897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.152145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.154759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b33ea-a928-43d9-a1ac-4a4caaadb66a None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.154759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b33ea-a928-43d9-a1ac-4a4caaadb66a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.155120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b33ea-a928-43d9-a1ac-4a4caaadb66a None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.156784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 477/961] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:07.162542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef546c-36b1-49b4-81c3-9ee64047eb2b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.162945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef546c-36b1-49b4-81c3-9ee64047eb2b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.163310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef546c-36b1-49b4-81c3-9ee64047eb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.163553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.165136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:07.178939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:07.180150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:58:07.181341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:07.188131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e7ef546c-36b1-49b4-81c3-9ee64047eb2b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:07.188131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e7ef546c-36b1-49b4-81c3-9ee64047eb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:07.189419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 485/962] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:07.190779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.258620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 478/963] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:07.260026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6dfb-5b97-4ab6-b21a-771860174379 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.260307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6dfb-5b97-4ab6-b21a-771860174379 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.260571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6dfb-5b97-4ab6-b21a-771860174379 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.261933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 479/964] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:07.267421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.267946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.268223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.286343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:07.287541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:07.304878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1a2b-f15d-4934-8fa4-cc6acb01ff12 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.342403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.343415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.343897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.344484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.345751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:07.346262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.347957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.357760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.364804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 480/965] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:07.365671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.380693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 486/966] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:07.387151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.393608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.488203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120360e8-2bce-4ba1-9296-6ef645b6c7ef None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.499522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e2a1-d0a2-4618-b5df-b62b486b209a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.524751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.525371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.525643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.525907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.526475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.527862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.535312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.535918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.536196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.536466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:07.536563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:07.537530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:07.538361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.542595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 481/967] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:07.543420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.607867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.613495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.614985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.619036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.648562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a891-c4f1-4c90-a7c3-da705c6dc0ec None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.690671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.691328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.691598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.691981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.693427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.694994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.717524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 487/968] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:07.717524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.737869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.sql.core [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:58:07.737869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 17:58:07.737869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [parameters: {'user_id': 'd399a16be8d34c3bba592f48ea9f5e46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 17:58:07.737869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80545) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result.read() Aug 09 17:58:07.746121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:58:07.747088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application yield resource Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise self.value Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application session.commit() Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:58:07.748037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:58:07.749181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:58:07.750063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application result.read() Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:58:07.752883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd399a16be8d34c3bba592f48ea9f5e46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:07.755774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 17:58:07.756743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 482/969] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Aug 9 17:58:07 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:07.757832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.838374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d9bd-33a7-451e-ad3d-853222d85d82 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.841385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.853985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e48104-ba04-4bb7-aec7-f18a62a696c7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:07.874123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.874784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.875089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.875331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.876508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:07.876811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.878170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.880376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:07.888338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 488/970] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:07.888338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.897069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:07.897697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:07.897970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:07.898237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:07.908489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:07.910854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:07.925866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:07.927089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:07.928206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:07.928206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:07.928206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:07.933052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 483/971] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:07.933897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:07.997492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e74e-89d8-442d-a3d5-33813ddb38c5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.034899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.035530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.035805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.036067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.044590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.046085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.048648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eaab8-ee15-4b43-9694-558ac3068d25 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:08.058014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:08.059005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 489/972] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:08.059958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.088575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.089102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.089376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.089642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.090296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:08.090587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.091975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.093909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.100778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 484/973] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:08.104281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.147773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.176204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d47ab1-6b96-49eb-8387-5c43e2dca10c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.209018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9526c-aacd-4935-8343-f56562fb5a76 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.211891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.212039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.212256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.212516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.213145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:08.213440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.214820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.227125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 490/974] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:08.227747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.251791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.252575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.253164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.253445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.254129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:08.254427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.255882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.291246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 485/975] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Aug 9 17:58:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:08.292067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.330925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.343678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a9b97-8a7e-4352-822f-981018847c87 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.385423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.386033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.386303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.386568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.387254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.388631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71852a8a-5efc-4c1f-814c-17e00ba37330 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.409941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.413951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777564-f81d-4456-ba99-c2c26b6fe102 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.451891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.452914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.454663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 491/976] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:08.455142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.459071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.459193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95caed-f649-4ac3-9b42-d0126c2ff67f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.459193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95caed-f649-4ac3-9b42-d0126c2ff67f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.459193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95caed-f649-4ac3-9b42-d0126c2ff67f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.464092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 492/977] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:08.466056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.466056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.466251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.470033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.471876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.489729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:08.491007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:08.493807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:08.494882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:08.495802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 486/978] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:08.502894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.593782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 493/979] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:08.617164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.638689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e0494-20ab-4ebf-929a-e3a15c28e00d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.677833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.678451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.678742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.679015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.679704 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:08.680417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.681874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.699497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 487/980] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:08.701070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:08.703149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.721912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:08.732583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eb0cb-87a0-4d28-8f72-4ee775061b85 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.773487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.774277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.774647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.775033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.775891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.777261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f9c31-8c0c-446c-a4d3-df191f56c688 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.806182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2110d6b-0378-446e-afdc-0a502ac12cea None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:08.839282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:08.839881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:08.840157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:08.840424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:08.841135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] RBAC: Adding query filter params (role.id=4ef7dec740ba4329ae1e895d9d41cc91, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=7c28473ae0554890a2b55941c15e94f4) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:08.841437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:08.842894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74161e-1793-4e22-88ca-d3bd5c2befbf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:08.849494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 488/981] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:08 2022] GET /identity/v3/role_assignments?user.id=7c28473ae0554890a2b55941c15e94f4&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=4ef7dec740ba4329ae1e895d9d41cc91 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:08.858874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 494/982] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:58:08 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:10.517267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daadb3e-614e-4bf1-9487-1f9066337dd2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.518070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daadb3e-614e-4bf1-9487-1f9066337dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.518070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daadb3e-614e-4bf1-9487-1f9066337dd2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.519575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 489/983] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:10.524660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.525076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.525238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.543547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:10.544715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.621119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 495/984] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:10.645076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daadb3e-614e-4bf1-9487-1f9066337dd2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:10.728549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:10.747208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daadb3e-614e-4bf1-9487-1f9066337dd2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:10.782911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:10.783547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.783824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.784092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.784670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:10.786035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-280b24aa-69f8-4a9d-998f-adf88065acc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:10.794741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:10.795736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:10.796632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:10.799036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 490/985] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:10.801888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:10.859340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:10.904712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d44a0c-0212-4bf8-bb67-f442ada98437 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:10.917949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d3ce-e159-4937-b2cd-cfe6992aabae None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.918224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d3ce-e159-4937-b2cd-cfe6992aabae None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.918493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d3ce-e159-4937-b2cd-cfe6992aabae None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.920016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 491/986] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:10.925266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9721d-7219-4530-adc2-b334d971608e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.925552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9721d-7219-4530-adc2-b334d971608e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.925820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9721d-7219-4530-adc2-b334d971608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.942217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:10.942853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.943129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.943394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.944009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:10.944304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:10.944955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d8c9721d-7219-4530-adc2-b334d971608e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:10.945650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3a84-86f6-4606-a914-62ca8a9fc545 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:10.946451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d8c9721d-7219-4530-adc2-b334d971608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.950382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 496/987] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:10.951685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae0ea-d975-452a-afdd-929e1fc3be28 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.951967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae0ea-d975-452a-afdd-929e1fc3be28 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.952235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae0ea-d975-452a-afdd-929e1fc3be28 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.953534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 497/988] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:10.957109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ae0ea-d975-452a-afdd-929e1fc3be28 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.014255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 492/989] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:11.018935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.018935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.018935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.034973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:11.036156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:11.065700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ae0ea-d975-452a-afdd-929e1fc3be28 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.085907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.099709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.100312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.100589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.100857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.101478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.101776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.102083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 493/990] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:11.102937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.103247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.108015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 498/991] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:11.114678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.117898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.210443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af11f9-f9c0-493c-be6e-b12a4c435d1b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.215760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7e4cd-823f-4b41-873a-b09fd71895fe None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.244479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.245101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.245375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.245649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.246218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.247603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.250829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.251514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.251791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.252060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.252757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] RBAC: Adding query filter params (interface=admin, service_id=9bb252b581034051a5f72e1dd6e3c84a, region_id=RegionOne) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.253060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.254397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:11.256077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:11.257082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.258002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 494/992] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:11.259005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.262270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 499/993] 10.222.0.36 () {56 vars in 1249 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/endpoints?service_id=9bb252b581034051a5f72e1dd6e3c84a&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:11.263143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.312213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.325942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.356526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101924-0eb4-4097-bb85-55deb0e5902c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.362815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb979da-180f-4c4e-b922-55fc1946bbb7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.392066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.392942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.393216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.393482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.394055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.395453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.397543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.398135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.398407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.398669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.399305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.399597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.400916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:11.403957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:11.405073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:11.410678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 495/994] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:11.411510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.411579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 500/995] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:11.411579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.512467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d571bb-162c-4705-aaa3-05c34c5e71a3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.512666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4726e3-5eff-4410-9f4b-dd521e4c344b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.526232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.546572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.547096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.547248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.548150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.548310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.548437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.548577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.548853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.548992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.549471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.549769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.550339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.551144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.551559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.556385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 501/996] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:11.562641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.563070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 496/997] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:11.563070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.679556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b22b7-3431-440d-977f-a2516ba0d92f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.682072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca266f-0dd8-4f90-b92e-29baad53c807 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.721608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.722006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.722212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.723810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.723919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.724009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.724093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.724093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.724229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] RBAC: Adding query filter params (interface=public, service_id=79aa8db46efe473191bee1f3b2b5d564, region_id=RegionOne) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.724229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.724370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] RBAC: Adding query filter params (name=image) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.724370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.725075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.725408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.731105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 502/998] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:11.731940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 497/999] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/endpoints?service_id=79aa8db46efe473191bee1f3b2b5d564&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:11.737563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.737972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.781232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.803118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.842807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559204-5031-46c2-9bb3-98bb15dbdedb None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.843241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef2352-801e-40cc-abba-62482ed5333c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.878820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.879153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.879434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.879681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.879788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.879949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.880042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.880220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.880550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.880924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.881222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.881915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af5b4b-c3b0-4a2b-9d25-c5c5f06800a5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.882555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def7cd-51cb-4bc7-874a-7c81c2b1b6c9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.887253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 503/1000] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:11.893580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 498/1001] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:13.602600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efee429-26c2-4523-98d1-a5eafe92e1a1 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.603163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efee429-26c2-4523-98d1-a5eafe92e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.603163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efee429-26c2-4523-98d1-a5eafe92e1a1 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.604639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 504/1002] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:13.609711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.610258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.610258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.629194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:13.630392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:13.699672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 499/1003] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:13.722641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efee429-26c2-4523-98d1-a5eafe92e1a1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:13.823901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efee429-26c2-4523-98d1-a5eafe92e1a1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:13.826231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:13.859248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:13.859903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.860182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.860490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.861153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:13.862522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-56216c39-11e2-463e-b902-f061f15d6a7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:13.871343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:13.872356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:58:13.873210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:13.874672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 505/1004] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Aug 9 17:58:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:13.879346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:13.926803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7de08-508a-4a22-9f78-92164c9fda4a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.927094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7de08-508a-4a22-9f78-92164c9fda4a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.931255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7de08-508a-4a22-9f78-92164c9fda4a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.931255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 506/1005] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:13.933542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362e04-d086-4f41-9af0-742793c00261 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.933810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362e04-d086-4f41-9af0-742793c00261 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.934080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362e04-d086-4f41-9af0-742793c00261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.936984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:13.953017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-17362e04-d086-4f41-9af0-742793c00261 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:13.954855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-17362e04-d086-4f41-9af0-742793c00261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:13.985828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d817-11d1-444a-be56-096a30fb5e4a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.020420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.021108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.021391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.021659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.022281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:14.022580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.023228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 507/1006] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:14.024014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.024472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d07c5-e77f-4e81-acac-034a998497c7 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.024758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d07c5-e77f-4e81-acac-034a998497c7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.025137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d07c5-e77f-4e81-acac-034a998497c7 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.026523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 508/1007] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:14.030923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 500/1008] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:14.031954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.032237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.032485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.034792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.050578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:14.051871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:14.117661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 509/1009] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:14.118517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.134684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d40fb-dbc8-4ebf-b47e-fb61f026f78a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.149275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:14.156253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:14.170357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.170983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.171261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.171530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.172769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:14.173071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.174549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.179233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 501/1010] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Aug 9 17:58:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:14.179999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.221920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8a3be-1b2f-455c-9320-c70f17db5323 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.258166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.258686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.259008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.259273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.260152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.261274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.282351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aec96-8d16-48a5-aecc-79d209caf793 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.315202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.315814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.316091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.316359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.316928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.318300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:14.326740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:14.327933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:14.330083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 502/1011] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:14.331104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.350813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 510/1012] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:58:14 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:14.351875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.381469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:14.442613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef195e7-da6b-4de0-b15a-c574c5819e4d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.480905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:14.499375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.499864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73605a-5c1f-4e34-a299-8d281950fc78 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.499996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.500249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.500648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.501308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.502841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.538453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.539076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.539344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.539609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.540230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:14.540526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.541881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.547071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 511/1013] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:14.553336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.572967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 503/1014] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:14.669073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb3bad-9b7c-4de4-b4f6-ba308117c429 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.737072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.737680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.738028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.738231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.738908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] RBAC: Adding query filter params (type=compute) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:14.739198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.740565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba00798-5d1f-46b9-9d1f-63d0a02753be None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:14.745994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 512/1015] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Aug 9 17:58:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:14.751517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:14.862490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c342-a9b2-495d-aa44-e0e2ae1fd8bd None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:14.865729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:14.897809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:14.898816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:14.899858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:14.900156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:14.901027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:14.902393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c15e7a-c30f-41fa-abba-945b18e9c103 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:15.013177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 504/1016] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:16.423646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf450ac-6a3f-4ecc-ad32-513182acf482 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.424211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf450ac-6a3f-4ecc-ad32-513182acf482 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.424211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf450ac-6a3f-4ecc-ad32-513182acf482 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.425667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 513/1017] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:16.430621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b6e7-5db6-45d8-844e-c331fd274f2c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.431038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b6e7-5db6-45d8-844e-c331fd274f2c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.431227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b6e7-5db6-45d8-844e-c331fd274f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.449553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-72e2b6e7-5db6-45d8-844e-c331fd274f2c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:16.450762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-72e2b6e7-5db6-45d8-844e-c331fd274f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.529234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 505/1018] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:16.552508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf450ac-6a3f-4ecc-ad32-513182acf482 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.564601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.650247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cc0-73eb-44e4-9fdd-0530861f4a44 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.650531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cc0-73eb-44e4-9fdd-0530861f4a44 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.650847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cc0-73eb-44e4-9fdd-0530861f4a44 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.652186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 506/1019] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:16.657220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.657505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.657996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.658362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf450ac-6a3f-4ecc-ad32-513182acf482 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.681878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:16.683163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.694339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.695007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.695459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.695586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] PATH_INFO: `/v3/services/image` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.696111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.697470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:16.708217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:16.709329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:16.710216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 514/1020] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:16.715387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.758907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 507/1021] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:16.782330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.792128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.799898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.824608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7ec13-21bd-4590-9c47-dd21a8404cbd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.858779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.859379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.859782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.859942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.860649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] RBAC: Adding query filter params (name=image) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:16.860950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.862350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.871954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 515/1022] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:16.872997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.890447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056248f1-b6fe-4bd9-9316-ff445b0da6cf None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.926949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.927577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.927852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.928121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.929256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.930722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.932528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 508/1023] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:16.939626 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.980026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8cd15-e692-49a7-91cd-81302c5dfaef None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.021133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.021742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.022334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.022610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.023427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] RBAC: Adding query filter params (type=image) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:17.023724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.025090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.027211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.028749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.035657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 516/1024] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:17.036769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.048133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1f11-7792-4193-8891-052f8c59383e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.083135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.083732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.084014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.084279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.084916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.086436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cfa6b-9521-44c6-8b75-09ee9e850998 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.137271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dba2c-1706-4123-ae5b-f088f3ee72ff None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.167034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 509/1025] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:17.168462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847071-a3a4-498b-a767-370826482c03 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.168742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847071-a3a4-498b-a767-370826482c03 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.169007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847071-a3a4-498b-a767-370826482c03 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.170194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 510/1026] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:17.176024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.176278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.180453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.184547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.185438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.185721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.185994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.186731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] RBAC: Adding query filter params (interface=public, service_id=88a14b19111d4a6d84f6f9d5f9af27e0, region_id=RegionOne) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:17.187052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.188698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.194819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 517/1027] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/endpoints?service_id=88a14b19111d4a6d84f6f9d5f9af27e0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:17.200358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.200582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:17.201782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:17.275361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.284367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 511/1028] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:17.308632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.320299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69126-ebbc-4744-a1e1-51d0173f2b9f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.358389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.359568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.360237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.360645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.361253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.362755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.371285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 518/1029] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:17.387592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.426601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63df854-5784-4a4d-9c26-a0e09e27df45 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.470535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.471206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.471489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.471758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.483487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:17.488727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:17.489812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:17.490686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 512/1030] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:17.496086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.606971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9fc5-cedc-4ccb-81c3-d3d9f00ffa6c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.633929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.642922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.643534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.643810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.644077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.649784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:17.649784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.649784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.655168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 519/1031] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:17.661296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.767648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb7ce-93c3-4e00-b76f-8fdae47f913f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.794223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.803446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.804085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.804362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.807066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.814717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.816193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:17.831610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:17.832599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:17.833483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:17.833483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:17.833483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:17.834797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 513/1032] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:17.838511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.944566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee00a9-7d6f-4a63-8b2c-cfa7fa698d21 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.979982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.980976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.981453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.982052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.983061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:17.983582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.985647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.997162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 520/1033] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:18.003326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:18.103825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf42f-fdea-4b58-bf6f-4edb5d8ccb35 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:18.137567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:18.138185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:18.138458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:18.138734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:18.147258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:18.148689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:18.159745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:18.160687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:18.161627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 514/1034] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:18.167870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:18.220926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:18.271204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e54af-9fb3-48c6-a78f-c7f47ff2b941 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:18.305397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:18.305997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:18.306273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:18.306539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:18.307282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:18.307582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:18.308995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:18.332561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 521/1035] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:18.334156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:18.374955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:18.439868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fbff1-048f-4c20-b559-7e1a48eb25e9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:18.479750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:18.480434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:18.480696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:18.480889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:18.553427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:18.554965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:18.595466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:18.661657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 515/1036] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:58:18.662743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:18.763716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2620-5f5e-4921-8601-49eb2f2974cd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:18.773939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:18.800072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:18.800701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:18.800975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:18.801241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:18.811206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:18.812672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:18.827053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:18.828567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:18.829629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:18.829629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:18.829629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:18.830864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 522/1037] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:18.834346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:18.933699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4af3c-9236-4426-a9f3-8ee4c0f5ff7d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:18.968560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:18.969745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:18.970011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:18.970268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:18.970927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:18.971223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:18.972599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:18.979147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:18.985691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 516/1038] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:18.991125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.091548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c513-1b36-4031-be8d-1188a3041fb9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.125682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.126313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.126589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.126888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.133536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.134997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-83693c1e-7a03-48e7-8ac0-5d1ff25f2152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:19.147051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:19.147978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:19.148902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:19.149917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 523/1039] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:19.153946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.258307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb67fd-7e04-43e6-a1b7-e0bc32454055 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.268259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44b9e-09a0-41cf-8fc6-478171fa496a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.268545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44b9e-09a0-41cf-8fc6-478171fa496a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.268817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44b9e-09a0-41cf-8fc6-478171fa496a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.270402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 524/1040] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:19.275398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.275677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.275947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.295808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:19.297524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:19.299334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.299938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.300214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.301195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.301195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:19.301450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.302889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.314423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:19.316175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 517/1041] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:19.321862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.373990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 525/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:19.383953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:19.398617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.428808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092382-7598-40d9-9f74-6ad402e9e463 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.466757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.467357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.467628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.467888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.468601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] RBAC: Adding query filter params (role.id=4ef7dec740ba4329ae1e895d9d41cc91, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:19.469189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.470672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba886b9-7ac3-49e7-9eb6-b99bf2f187c0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.476200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 518/1043] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=4ef7dec740ba4329ae1e895d9d41cc91 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:19.477415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656de6-7d14-4e52-abff-3f6dcb283287 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.477697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656de6-7d14-4e52-abff-3f6dcb283287 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.477964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd656de6-7d14-4e52-abff-3f6dcb283287 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.479187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 519/1044] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:19.488145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.488782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.488782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.508462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:19.509584 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:19.519232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d67302-5694-4be5-8de3-f40efd3db44c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.563864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.564705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.565177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.565572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.567151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.568920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.570603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 526/1045] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:19.576315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.597920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 520/1046] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:19.601409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:19.614375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.677760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:19.692036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc86e1-bbc7-4a9d-aa79-a1d2c352f012 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.727804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3f3bb-2c2a-483d-a93f-9e317ea04bf9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.730838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.731446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.731719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.731987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.732633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.734057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.764023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.764550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.764822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.765086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] PATH_INFO: `/v3/services/image` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.765769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.767553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:19.776650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:19.777801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:19.779364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 521/1047] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:19.784235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:19.824974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 527/1048] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:58:19 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:19.901223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee5139-afb8-474f-a734-951f0996ec27 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:19.918913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:19.939076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:19.939753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:19.940040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:19.940311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:19.940952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] RBAC: Adding query filter params (name=image) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:19.941325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:19.942828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:19.948777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 522/1049] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:19.956157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:20.040243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:20.068934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf6b7c-af96-4889-86dd-d3be8110dc4b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:20.103900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:20.104513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:20.104852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:20.105131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:20.105759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] RBAC: Adding query filter params (type=image) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:20.106059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:20.107462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398316ae-cb4e-48dd-825a-7b17acb8d915 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:20.117919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 528/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:20.119171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:20.223063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54744d6-1917-4851-be31-5317d9328cfa None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:20.256046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:20.256643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:20.256915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:20.257179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:20.257822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:20.259210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a2ad-a8fe-43d9-b694-fa0fce372432 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:20.343223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 523/1051] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:21.518022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb701-7e2d-4294-8576-b2d6dadaddef None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.518578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb701-7e2d-4294-8576-b2d6dadaddef None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.518578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb701-7e2d-4294-8576-b2d6dadaddef None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.522747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 529/1052] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:21.525115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.525522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.525700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.544229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:21.545429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:21.576050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:21.623782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 524/1053] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:21.645984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abb701-7e2d-4294-8576-b2d6dadaddef None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:21.748141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abb701-7e2d-4294-8576-b2d6dadaddef None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:21.783204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:21.783831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.784108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.784375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.791244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:21.795445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a66c69c9-8b17-4775-95bb-ac47069451a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:21.800265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:21.802977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:21.803879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 530/1054] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:58:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:21.807719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:21.906663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc851-5d69-4667-a748-188049b572fb None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.906955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc851-5d69-4667-a748-188049b572fb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.907189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc851-5d69-4667-a748-188049b572fb None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.909177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 531/1055] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:21.909490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53235e06-0d8e-4c96-8b26-44bc33f83355 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:21.914019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.914288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.914560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.932094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:21.933258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:21.944971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:21.945543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:21.945814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:21.946072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:21.946740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:21.947029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:21.948369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:21.957338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 525/1056] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:21.963032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:21.990525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.007027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 532/1057] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:22.030982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.069908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ae10f-0b79-4153-ade6-0540439f469b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.106468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.107104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.107377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.107637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.119012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.120445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.132519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05b3c-6dad-46ac-884d-1d52fa75728a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:22.135005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:22.136131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:22.137297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:22.137297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:22.137297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:22.137297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 526/1058] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:22.141711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.162893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.172370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.173079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.173965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.173965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.183816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:22.189210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:22.190171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:22.191054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:22.195444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 533/1059] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:22.196287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.226169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.252917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6241-8c8d-4561-a985-3bce16d23ce0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.288440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.288958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.289230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.289492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.290119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:22.290412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.291805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.303369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44a6ed-0828-4c45-a6f0-86957d841aeb None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.304225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 527/1060] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:22.309495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.340314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.340917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.341192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.341450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.342096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:22.342389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.344460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.357942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 534/1061] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:22.358930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.408924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.417120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bd30a-67ab-4b4c-972d-b4a00dedd764 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.451776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.452386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.452662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.456871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.457469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7552a4-d1b8-4810-86ee-99196c22c7ae None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.458311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.465324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.466808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-90838b4b-0596-484d-b598-bdec3ad7f27e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:22.479483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:22.480648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:22.481762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 528/1062] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:58:22.482374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5419a-f3f6-4f3f-9b9e-5ecaa422e8be None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.482374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5419a-f3f6-4f3f-9b9e-5ecaa422e8be None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.482509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5419a-f3f6-4f3f-9b9e-5ecaa422e8be None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.485605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 529/1063] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:22.487034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5419a-f3f6-4f3f-9b9e-5ecaa422e8be None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.497973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.498596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.498897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.499167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] PATH_INFO: `/v3/users/74889cee25b84a659f6146a48ea9b1a2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.512121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] RBAC: Authorizing `identity:get_user(user_id=74889cee25b84a659f6146a48ea9b1a2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.513549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a14611-8547-4e03-9140-09e37789ee11 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.515175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 535/1064] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/users/74889cee25b84a659f6146a48ea9b1a2 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:22.516817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.517354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.517619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.534833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:22.535998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:22.600722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5419a-f3f6-4f3f-9b9e-5ecaa422e8be None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.609159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 536/1065] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:22.610938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.653638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.654237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.654510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.654808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.655549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:22.656115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.656572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.658225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.681131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 530/1066] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:22.681983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.725085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42706e08-f159-49b5-ab9e-b94ec8f98222 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.744500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.762319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.762952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.763230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.763495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.789676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf39c9c-1d9e-47f5-b794-edb45479c558 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:22.829659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:22.830797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:22.831632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:22.832364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:22.840793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.842276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.883970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:22.907787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:22.909233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:22.943803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 537/1067] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:22.944639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:22.965011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.010308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 531/1068] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:23.011167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.049666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9313-1fe8-485e-9329-700a4e9ee43a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.083855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.084477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.084756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.085025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.091422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.096132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.097575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.107958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a533-2c9a-4534-be50-f42e94406dbc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:23.111785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:23.112987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:23.114026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:23.114026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:23.114026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:23.114026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 538/1069] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:23.114026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.144443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.144985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.145251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.145517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.146598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.148065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.149672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 532/1070] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:23.151529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.175143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.214683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48432e1-8c60-4f33-bfb0-6c98891e60e0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.248843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.249457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.249594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009de1f-1e2c-493e-8bd5-49ccaeb332f9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.250323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.250681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.261173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.262594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:23.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:23.277516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:23.278422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:23.278422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:23.278422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:23.278422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 539/1071] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:23.278422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.288542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.289140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.289414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.289679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.290308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:23.290602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.291993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.308425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 533/1072] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:23.308425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.326949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.382461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a24d93-ce17-4bad-ae82-f0ac43c184d4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.385107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.402597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cd37a-5691-4c4b-9302-315f403e0a77 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.417180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.417788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.418066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.418335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.419081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=cdd66e670442454f920ff201b9487ea2, user.id=74889cee25b84a659f6146a48ea9b1a2) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:23.419386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.420808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.426619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 540/1073] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/role_assignments?user.id=74889cee25b84a659f6146a48ea9b1a2&scope.project.id=cdd66e670442454f920ff201b9487ea2&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:23.427427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.437014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.437713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.437949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.438216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.438900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:23.439206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.440635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.454757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 534/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:23.560252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddbe3e-a7ce-4dbf-9a05-0c9acb705296 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.569268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:23.601589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.602237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.602518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.602818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.611887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.613365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:23.625779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:23.634829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:23.635945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 541/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:23.636555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.796649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49905d88-23e6-4558-a961-24c8f9c7aeab None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.830162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.830769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.831048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.831315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.832053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:23.832450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.834222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa105a43-9653-48af-bd4d-903fa68024d1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:23.846335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 535/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:23.852300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:23.955048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddcc9a-f6cf-4c9d-84d2-85ff120966ce None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:23.988139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:23.988782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:23.989056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:23.989324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/4ef7dec740ba4329ae1e895d9d41cc91` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:23.991434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=89d8460000674bcead45625785d81867, role_id=4ef7dec740ba4329ae1e895d9d41cc91)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:23.992849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415f914-5d49-43b3-947e-f3cbb6f67872 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:24.068811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 542/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:58:23 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/89d8460000674bcead45625785d81867/roles/4ef7dec740ba4329ae1e895d9d41cc91 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:25.443953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f75cc2-1589-4f5d-a386-e14685a90fe2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:25.444520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f75cc2-1589-4f5d-a386-e14685a90fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:25.444520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f75cc2-1589-4f5d-a386-e14685a90fe2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:25.445806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 536/1078] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:25.451194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:25.451632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:25.451734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:25.471016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:25.472232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:25.491957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:25.538681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 543/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:25.560569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f75cc2-1589-4f5d-a386-e14685a90fe2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:25.563502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:25.668120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f75cc2-1589-4f5d-a386-e14685a90fe2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:25.701492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:25.702026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:25.702301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:25.702595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:25.713484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:25.718132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:25.719648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:25.720478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:25.723017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 537/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:25.725769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:25.821864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac8fb-0f11-49c4-be8e-e2bcbe0a0fc2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:25.855375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:25.855977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:25.856257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:25.856530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:25.857182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:25.857485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:25.858937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:25.867526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 544/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:25.873098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:25.932917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:25.971771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373b63f-d768-4a78-a5c1-205c19c658db None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.004567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.005186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.005461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.005729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] PATH_INFO: `/v3/users/74889cee25b84a659f6146a48ea9b1a2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.006856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] RBAC: Authorizing `identity:get_user(user_id=74889cee25b84a659f6146a48ea9b1a2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.008272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d86998-63b2-450b-856a-362fad7b21dd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.009838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 538/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:25 2022] GET /identity/v3/users/74889cee25b84a659f6146a48ea9b1a2 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:26.016270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.036253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:26.076694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5a35-02f3-4448-9dc5-028b6b2b12a8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.076971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5a35-02f3-4448-9dc5-028b6b2b12a8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.077239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5a35-02f3-4448-9dc5-028b6b2b12a8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.079299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 539/1083] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:58:26.084344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.084624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.084900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.104267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:26.105454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:26.119543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607ea5-4462-400b-a118-cfdad3da4db6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.153367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.153982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.154264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.154528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.155678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.157736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.159528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 545/1084] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:26.165086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.172832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 540/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:26.195681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.248078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:26.251800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:26.262596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3ab2-98d8-4da7-8cb7-ad194ff85586 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.292682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dda30-8066-4f06-bae0-a57755776b3f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.298656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.299345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.299624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.299896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2/users/74889cee25b84a659f6146a48ea9b1a2/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.301911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdd66e670442454f920ff201b9487ea2, user_id=74889cee25b84a659f6146a48ea9b1a2, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.303366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.325759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.326372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.326645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.326963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.337506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:26.342194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:26.344045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:26.344898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 541/1086] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:26.350034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.359588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 546/1087] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:58:26 2022] PUT /identity/v3/projects/cdd66e670442454f920ff201b9487ea2/users/74889cee25b84a659f6146a48ea9b1a2/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:26.455945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997e057-29e0-4db9-884a-0122533d491f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.481958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:26.496534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.497140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.497416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.497685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.498450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:26.498687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.500868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.510870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 542/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:26.519936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.622118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519917d6-bfe7-46a9-8157-023c2a487705 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.656975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.657601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.657875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.658136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.667901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.669322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.676998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:26.684823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:26.685839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:26.686763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:26.686763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:26.686763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:26.686763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 547/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:26.695933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.796696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39936a-afe1-449a-a5a8-c622efa31653 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.830963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.831562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.831837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.832104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.832733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:26.833032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.834387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:26.846343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 543/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:26.851937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:26.948927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b5669-c5ed-4b3a-b22b-604c09f7551a None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:26.982058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:26.982805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:26.983085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:26.983350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:26.990692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:26.992128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:58:27.002804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:27.003879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:27.004805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 548/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Aug 9 17:58:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:27.009504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.053435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:27.109069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20543ae-0d67-48a2-8f75-6557b9aa67bd None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:27.143253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:27.143835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:27.144106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:27.144368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:27.145070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:27.145363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:27.146790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.163980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 544/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:27.169826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.227199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:27.268617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304b822-5c6d-4f2d-b466-db448b6ea1fd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:27.300493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:27.301078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:27.301356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:27.301623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:27.366384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:27.367781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.423577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:27.468430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 549/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:27.474305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.569376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:27.572448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fffd3e-61f5-44a3-9a71-8402822cf1d5 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:27.605404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:27.606020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:27.606293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:27.606554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:27.617175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:27.618541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:27.633410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:27.634446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:58:27.635404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:27.635404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:58:27.635404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:27.638870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 545/1094] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:27.640325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.735763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af2-001b-4004-8ac1-918361d13b22 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:27.768743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:27.769333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:27.769607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:27.769873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:27.770502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:27.770837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:27.772193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.783428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 550/1095] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:27.789014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.887856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b56a-fc06-496a-8e6c-b6f2f7d651e1 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:27.920966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:27.922150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:27.922439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:27.922806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:27.929350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:27.930815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:58:27.941802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:27.942683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:58:27.943601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:27.946856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 546/1096] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:27.948833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:27.967857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.048170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc369448-a87e-4b27-811f-f6b135ecfa26 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.081549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.082139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.082423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.082668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.083364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:28.083656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.085060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b370-49a6-4651-a0f7-86f8bc26f696 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.096921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 551/1097] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 17:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:28.102677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.117914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.200832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ca44a-a454-4273-a272-2126649ec25b None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.233090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.234197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.234588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.234886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.235604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] RBAC: Adding query filter params (role.id=4ef7dec740ba4329ae1e895d9d41cc91, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=89d8460000674bcead45625785d81867) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:28.235903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.237321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c52e2-eb9d-468a-824b-b852d9d9decf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.242984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 547/1098] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/role_assignments?user.id=89d8460000674bcead45625785d81867&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=4ef7dec740ba4329ae1e895d9d41cc91 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:28.379422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ff3f-e325-439e-997b-27485e7bf18e None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.379708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ff3f-e325-439e-997b-27485e7bf18e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.380008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ff3f-e325-439e-997b-27485e7bf18e None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.381338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 552/1099] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:28.386905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.387189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.387465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.407690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:28.408886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:28.480654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 548/1100] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:28.505898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9ff3f-e325-439e-997b-27485e7bf18e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.587674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.612133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9ff3f-e325-439e-997b-27485e7bf18e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.647641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.648262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.648517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.648782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.659928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:28.664573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:28.665579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:28.666428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 553/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:28.672794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.681086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.776726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd022aa-91f7-4ab0-ad80-1cabbca0ffc0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.813867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.814467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.814770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.815043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.815699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:28.815995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.817368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.825525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 549/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:28.831971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.933661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544111f-bc39-424e-bc58-d04e8afb6e2f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.968536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.969319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.969591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.969858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] PATH_INFO: `/v3/users/74889cee25b84a659f6146a48ea9b1a2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.971008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] RBAC: Authorizing `identity:get_user(user_id=74889cee25b84a659f6146a48ea9b1a2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.972429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.973991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 554/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/users/74889cee25b84a659f6146a48ea9b1a2 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:28.980176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.048479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:29.082049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ba600-0ae0-4424-b6f6-3da04f6a1c3c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.117313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.117905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.118176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.118439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.184498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.185924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.248273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:29.281502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 550/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:29.287000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.295543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:29.383767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1ae43-c481-4139-bd0b-b3166ddce833 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.415897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.416506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.416782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.417049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.418145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.419596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be350-30a8-4518-a8be-3d9835a881f3 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.422175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 555/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:29 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:29.427891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.524094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751bd5-d85b-40b1-af6f-bbe05662458f None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.556920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.557511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.557787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.558054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.558792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=cdd66e670442454f920ff201b9487ea2, user.id=74889cee25b84a659f6146a48ea9b1a2) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:29.559097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.567005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e17816-2c9d-4efa-8d67-08744282ba03 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.567005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 551/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:29 2022] GET /identity/v3/role_assignments?user.id=74889cee25b84a659f6146a48ea9b1a2&scope.project.id=cdd66e670442454f920ff201b9487ea2&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:31.610829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44db7d-ac6a-4fa8-957d-42a57df30a11 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.611355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44db7d-ac6a-4fa8-957d-42a57df30a11 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.611355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44db7d-ac6a-4fa8-957d-42a57df30a11 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.612684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 556/1107] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:31.617908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.618337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.618437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.636507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:31.637674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:31.663177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:31.703067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 552/1108] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:31.728641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44db7d-ac6a-4fa8-957d-42a57df30a11 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:31.803121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:31.829403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44db7d-ac6a-4fa8-957d-42a57df30a11 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:31.861318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:31.862107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.862388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.862653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.863776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:31.865194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fac5b-232c-4fbe-929b-3388f3a8343b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:31.866805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 557/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:31.872572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:31.967446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b51dd-a6ff-43e5-9676-bdbacf479974 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.000528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.001108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.001385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.001653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.002288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.003785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baae46-5548-4931-9108-82203ce19c5d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.090280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 553/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:58:31 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:34.148445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f6ee-61af-4574-b894-5656ef02f954 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.149007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f6ee-61af-4574-b894-5656ef02f954 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.149007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f6ee-61af-4574-b894-5656ef02f954 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.150296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 558/1111] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:34.155438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.156032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.156032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.174335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:34.175550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:34.188803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.241033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 554/1112] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:34.264954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5f6ee-61af-4574-b894-5656ef02f954 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.363316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5f6ee-61af-4574-b894-5656ef02f954 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.396128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.396735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.397009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.397277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] PATH_INFO: `/v3/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.398689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] RBAC: Authorizing `identity:get_role(role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.400021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.401327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 559/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:34 2022] GET /identity/v3/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:34.407454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.503717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94a78f-232f-4329-805c-b26ff7757006 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.536114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.536743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.537023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.537295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] PATH_INFO: `/v3/users/699fb588630e4772856ea05e09a30ef2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.550533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] RBAC: Authorizing `identity:get_user(user_id=699fb588630e4772856ea05e09a30ef2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.552000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.553600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 555/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:34 2022] GET /identity/v3/users/699fb588630e4772856ea05e09a30ef2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:34.559471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.572334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.659062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11d9c3-1d71-4f14-bfa5-695ba4c1caed None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.693000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.693589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.693861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.694129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.759802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.761230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.772251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.857727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 560/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:34.864210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.872640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.960522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fe2fd-2b71-4e27-bcce-18efde177546 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.992973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.993575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.993857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.994124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.995239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.996651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91392347-d47b-4abf-8072-b8c1faeed3ab None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.998252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 556/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:34 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:35.004949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:35.099081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ba2c7-027c-4bcd-912c-b92a5ed4b2bb None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:35.109788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:35.132492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:35.133074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:35.133351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:35.133621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:35.134350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=cdd66e670442454f920ff201b9487ea2, user.id=699fb588630e4772856ea05e09a30ef2) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:35.134662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:35.136775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848194f9-64d9-4562-90fe-1b6114ba4da5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:35.142717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 561/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:35 2022] GET /identity/v3/role_assignments?user.id=699fb588630e4772856ea05e09a30ef2&scope.project.id=cdd66e670442454f920ff201b9487ea2&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:37.110473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526685-2978-4d68-8393-c9f68e16036e None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.111085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526685-2978-4d68-8393-c9f68e16036e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.111085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526685-2978-4d68-8393-c9f68e16036e None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.112341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 557/1118] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:37.117409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.117843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.117942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.136652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:37.137819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:37.203082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 562/1119] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:37.224837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed526685-2978-4d68-8393-c9f68e16036e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.298412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:37.323080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed526685-2978-4d68-8393-c9f68e16036e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.354942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.355543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.355811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.356070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] PATH_INFO: `/v3/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.357419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] RBAC: Authorizing `identity:get_role(role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.359387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.360719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 558/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:37 2022] GET /identity/v3/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:37.366533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.462259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb688f-5efd-4a86-8e76-71a1868f4fc5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.464555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:37.496916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.497529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.497803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.498070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] PATH_INFO: `/v3/users/699fb588630e4772856ea05e09a30ef2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.499779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] RBAC: Authorizing `identity:get_user(user_id=699fb588630e4772856ea05e09a30ef2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.501362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.502959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 563/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:37 2022] GET /identity/v3/users/699fb588630e4772856ea05e09a30ef2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:37.509018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.603729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e98a31-7a4b-4472-af9e-0fe2abd7e559 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.636114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.636655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.636930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.637196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.638288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.639775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc1a0c-8394-4421-b4c4-b16f6ddcd270 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.641639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 559/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:37 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:37.647632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.744255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9a2a-af65-441c-a1b4-8fdd664aa029 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.776706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.777342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.777619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.777890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2/users/699fb588630e4772856ea05e09a30ef2/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.780604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] RBAC: Authorizing `identity:create_grant(project_id=cdd66e670442454f920ff201b9487ea2, user_id=699fb588630e4772856ea05e09a30ef2, role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.782020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49b32b-8868-47d9-886b-80f361d6791e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.867365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 564/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:58:37 2022] PUT /identity/v3/projects/cdd66e670442454f920ff201b9487ea2/users/699fb588630e4772856ea05e09a30ef2/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:39.867077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380af73-d74a-4d9a-af4e-a9b84982c2cf None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:39.867605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380af73-d74a-4d9a-af4e-a9b84982c2cf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:39.867605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380af73-d74a-4d9a-af4e-a9b84982c2cf None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:39.868917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 560/1124] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:39.874602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:39.875072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:39.875168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:39.893407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:39.894579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:39.919375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:39.960328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 565/1125] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:39.984766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380af73-d74a-4d9a-af4e-a9b84982c2cf None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:39.996674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:40.082886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380af73-d74a-4d9a-af4e-a9b84982c2cf None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:40.116603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:40.117271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:40.117493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:40.117760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] PATH_INFO: `/v3/roles/3900c649bfe64bfe8aeaa413b2df804f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:40.119156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] RBAC: Authorizing `identity:get_role(role_id=3900c649bfe64bfe8aeaa413b2df804f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:40.120515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:40.121813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 561/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:39 2022] GET /identity/v3/roles/3900c649bfe64bfe8aeaa413b2df804f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:40.127729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:40.222634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aff22-a63b-437a-955a-79aa5f645ed1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:40.255400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:40.256016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:40.256296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:40.256568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] PATH_INFO: `/v3/users/699fb588630e4772856ea05e09a30ef2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:40.257672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] RBAC: Authorizing `identity:get_user(user_id=699fb588630e4772856ea05e09a30ef2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:40.259135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:40.260678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 566/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:40 2022] GET /identity/v3/users/699fb588630e4772856ea05e09a30ef2 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:40.267279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:40.341749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:40.362799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff3da6-3781-49d9-834a-e102ca57cb6c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:40.395786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:40.396316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:40.396592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:40.396862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:40.461709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:40.463199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:40.537281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:40.558360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 562/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:40.564523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:40.594599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:40.661940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86802-8b8a-49a2-b9fe-a120e76be1fc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:40.694767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:40.695380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:40.695657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:40.695928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] PATH_INFO: `/v3/projects/cdd66e670442454f920ff201b9487ea2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:40.697024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd66e670442454f920ff201b9487ea2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:40.698443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be744b-f11f-485c-9fa1-e8595e060ed2 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:40.700088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 567/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:40 2022] GET /identity/v3/projects/cdd66e670442454f920ff201b9487ea2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:40.706897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:40.801498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1431aa-9066-4124-ba6f-61d2211e6eac None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:40.834465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:40.835065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:40.835337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:40.835600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:40.836294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] RBAC: Adding query filter params (role.id=3900c649bfe64bfe8aeaa413b2df804f, scope.project.id=cdd66e670442454f920ff201b9487ea2, user.id=699fb588630e4772856ea05e09a30ef2) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:40.836589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:40.837990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9fe1c-64ae-4d90-9f03-e7f4533a5a49 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:40.843905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 563/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:40 2022] GET /identity/v3/role_assignments?user.id=699fb588630e4772856ea05e09a30ef2&scope.project.id=cdd66e670442454f920ff201b9487ea2&role.id=3900c649bfe64bfe8aeaa413b2df804f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:42.877039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6901-b7f5-43db-8bd2-fbc838ea63de None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:42.877606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6901-b7f5-43db-8bd2-fbc838ea63de None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:42.877606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6901-b7f5-43db-8bd2-fbc838ea63de None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:42.878938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 568/1131] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:42.884151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:42.884557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:42.884693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:42.903416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:42.905590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:42.940189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:42.970689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 564/1132] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:42.994110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf6901-b7f5-43db-8bd2-fbc838ea63de None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:43.089468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:43.094624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf6901-b7f5-43db-8bd2-fbc838ea63de None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:43.128854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:43.129398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:43.129676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:43.129945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:43.131077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:43.132514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f033a-0707-42b5-b367-6896f54a2828 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:43.134084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 569/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:43.140114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:43.237155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8fb0-ce54-40d0-8f8a-08068758f308 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:43.270644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:43.271270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:43.271548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:43.271817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:43.272485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:43.273977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6141-1161-494e-b750-de4f93d5851d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:43.354068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 565/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:58:43 2022] POST /identity/v3/projects => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:45.386227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5147-bc1f-4681-be5b-9e0502cf6b79 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.386784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5147-bc1f-4681-be5b-9e0502cf6b79 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.386784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee5147-bc1f-4681-be5b-9e0502cf6b79 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.388115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 570/1135] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:45.394226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.394688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.394798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.412975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:45.414131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:45.430597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:45.480911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 566/1136] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:45.506445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee5147-bc1f-4681-be5b-9e0502cf6b79 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:45.603714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee5147-bc1f-4681-be5b-9e0502cf6b79 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:45.642417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:45.642417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.642417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.642417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.642979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:45.642979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfce65d-1022-452d-9e88-802eb0c46fba None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:45.643222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 571/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 17:58:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:45.648983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:45.744971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb8ea-44f0-45f4-97a9-81e25be1b5fc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:45.780289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:45.780641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.780914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.781181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.781815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:45.783273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b436f5-c933-4902-ba10-d8da4f11d90d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:45.861299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 567/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:58:45 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:47.919326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad22813-6502-417c-b746-574bc48d490a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.919941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad22813-6502-417c-b746-574bc48d490a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.920060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad22813-6502-417c-b746-574bc48d490a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.921554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 572/1139] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:47.926910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.927319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.927452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.945056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:47.946232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:47.948942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:48.010790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 568/1140] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:48.035152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad22813-6502-417c-b746-574bc48d490a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:48.068672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:48.133589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad22813-6502-417c-b746-574bc48d490a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:48.166367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:48.166997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:48.167277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:48.167541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:48.177319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:48.181861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:48.182909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:48.183918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 573/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:48.190244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:48.285156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364b6ec-959b-4253-bc78-a949e47ad2d8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:48.317067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:48.317649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:48.317921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:48.318182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:48.318846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:48.319140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:48.320482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:48.328834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 569/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:48.334658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:48.426272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:48.432692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1756-5dd8-4007-8b85-6452bb596376 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:48.464637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:48.465260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:48.465535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:48.465826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] PATH_INFO: `/v3/users/c2d746cad0db4722982f81e69710f7f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:48.479581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d746cad0db4722982f81e69710f7f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:48.481001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:48.483165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 574/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/users/c2d746cad0db4722982f81e69710f7f6 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:48.489038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:48.491875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:48.586475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79e67a-8758-4d8e-b455-9170b9190d47 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:48.619063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:48.619650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:48.619920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:48.620183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:48.685611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:48.687109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:48.688973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:48.783899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 570/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:48.789798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:48.885607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb611-994c-4ddb-8166-221581cd6f36 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:48.918400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:48.919037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:48.921614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:48.921614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] PATH_INFO: `/v3/projects/ca839d2e70a0420a986777c81698ead8` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:48.921614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ca839d2e70a0420a986777c81698ead8)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:48.922426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d5072-dd8f-4a82-9e63-16441af9f8f4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:48.923748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 575/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/projects/ca839d2e70a0420a986777c81698ead8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:48.929863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:49.026054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bbbcd-5a6c-4fdc-beab-385caed43097 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:49.027541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:49.059529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:49.060112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:49.060436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:49.060765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:49.061770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ca839d2e70a0420a986777c81698ead8, user.id=c2d746cad0db4722982f81e69710f7f6) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:49.062131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:49.063820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c36a-a671-4670-b38d-b42d321849ad None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:49.069259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 571/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:48 2022] GET /identity/v3/role_assignments?user.id=c2d746cad0db4722982f81e69710f7f6&scope.project.id=ca839d2e70a0420a986777c81698ead8&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:51.064470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a297ca-03bc-40ec-a41e-11f2fb4ca3b3 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.065039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a297ca-03bc-40ec-a41e-11f2fb4ca3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.065039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a297ca-03bc-40ec-a41e-11f2fb4ca3b3 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.066334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 576/1147] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:51.071341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.071759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.071890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.089771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:51.090970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:51.155313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 572/1148] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:51.177418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a297ca-03bc-40ec-a41e-11f2fb4ca3b3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.199045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:51.277373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a297ca-03bc-40ec-a41e-11f2fb4ca3b3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.309704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.310962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.311238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.311506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.321223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:51.326494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:51.327632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:51.328672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 577/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:51.333689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.422734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:51.428163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f2cb-909d-4d7e-88fb-463656dbdf19 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.461867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.462466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.462759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.463028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.463873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:51.463972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.465313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.474850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 573/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:51.480722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.559201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:51.577872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462c805-357d-405c-bf0b-cb2de9f3a763 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.611005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.611611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.611886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.612154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] PATH_INFO: `/v3/users/c2d746cad0db4722982f81e69710f7f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.613253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] RBAC: Authorizing `identity:get_user(user_id=c2d746cad0db4722982f81e69710f7f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.614675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.616254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 578/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:51 2022] GET /identity/v3/users/c2d746cad0db4722982f81e69710f7f6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:51.621940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.718135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c17bb-6bab-4149-bfcb-0ec520341e84 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.749540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.750145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.750416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.750681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] PATH_INFO: `/v3/projects/ca839d2e70a0420a986777c81698ead8` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.751800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] RBAC: Authorizing `identity:get_project(project_id=ca839d2e70a0420a986777c81698ead8)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.753209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d000d58-a621-49b0-b590-f1c485aa245c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.754967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 574/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:51 2022] GET /identity/v3/projects/ca839d2e70a0420a986777c81698ead8 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:51.760769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.858731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2d5a-2b54-4f84-a39d-6760d7361aed None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.893196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.893833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.894111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.894381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] PATH_INFO: `/v3/projects/ca839d2e70a0420a986777c81698ead8/users/c2d746cad0db4722982f81e69710f7f6/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.896288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca839d2e70a0420a986777c81698ead8, user_id=c2d746cad0db4722982f81e69710f7f6, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.897713 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9b74e-e7fd-44fd-971f-197d68941b7d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.009352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 579/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:58:51 2022] PUT /identity/v3/projects/ca839d2e70a0420a986777c81698ead8/users/c2d746cad0db4722982f81e69710f7f6/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:53.982678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a283c-6117-4d53-a864-02660cc23aad None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.983264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a283c-6117-4d53-a864-02660cc23aad None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.983264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a283c-6117-4d53-a864-02660cc23aad None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.984591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 575/1154] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:53.989651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.990036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.990194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.007702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:54.008873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:54.010792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:54.073528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 580/1155] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:54.098038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a283c-6117-4d53-a864-02660cc23aad None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:54.125325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:54.197789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a283c-6117-4d53-a864-02660cc23aad None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:54.230925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:54.231536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:54.231948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:54.232229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.242339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:58:54.246862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:54.248706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:58:54.249817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 576/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:54.254581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:54.349567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e3db9-052b-4e27-9b65-be5617132301 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:54.382336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:54.382932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:54.383209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:54.383477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.384120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:54.384415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:54.385763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:54.391605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:54.394961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 581/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:54.400913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:54.485909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:54.500753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08ae52-3d2a-4c06-b60f-233aaeae4a2c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:54.532903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:54.533516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:54.533785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:54.534050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] PATH_INFO: `/v3/users/c2d746cad0db4722982f81e69710f7f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.535167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d746cad0db4722982f81e69710f7f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:54.536587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:54.538164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 577/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/users/c2d746cad0db4722982f81e69710f7f6 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:54.543977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:54.642919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bfdb-fa1e-4eff-ba52-fd2f5931ba54 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:54.675408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:54.676000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:54.676277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:54.676541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.738887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:54.741002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:54.742449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:54.836893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 582/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:54.843457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:54.939833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867c9d-7dee-4e63-875d-61ba8889b506 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:54.972068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:54.972676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:54.972952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:54.973219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] PATH_INFO: `/v3/projects/ca839d2e70a0420a986777c81698ead8` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.974316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca839d2e70a0420a986777c81698ead8)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:54.975775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc073fc-5e72-4306-bbbc-4f1826727cc8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:54.977370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 578/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/projects/ca839d2e70a0420a986777c81698ead8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:54.984066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:55.076517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:55.079958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87a44d-8bca-4ae6-829b-c21c121b8983 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:55.112651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:55.113177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:55.113452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:55.113716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:55.114418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ca839d2e70a0420a986777c81698ead8, user.id=c2d746cad0db4722982f81e69710f7f6) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:55.114740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:55.116253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768acc8-cf69-4a11-8282-1b37eff89fcf None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:55.121688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 583/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:58:54 2022] GET /identity/v3/role_assignments?user.id=c2d746cad0db4722982f81e69710f7f6&scope.project.id=ca839d2e70a0420a986777c81698ead8&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:57.117875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f04969-fd02-4d8b-8ab6-8a71b2d06ec9 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:57.118409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f04969-fd02-4d8b-8ab6-8a71b2d06ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:57.118409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f04969-fd02-4d8b-8ab6-8a71b2d06ec9 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:57.119777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 579/1162] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:57.125510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:57.125954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:57.126057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:57.144511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:57.145678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:57.209866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 584/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:58:57.231539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f04969-fd02-4d8b-8ab6-8a71b2d06ec9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:57.256068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:57.332051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f04969-fd02-4d8b-8ab6-8a71b2d06ec9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:57.363742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:57.364326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:57.364602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:57.364870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:57.429574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:57.431164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:57.454966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:57.529002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 580/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:58:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:58:57.534999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:57.623792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:57.630239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612445b5-acce-4e42-985b-1b4bb77f7803 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:57.663080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:57.663693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:57.663970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:57.664237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:57.671203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:57.672640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-53869f40-30d2-4690-830a-ffceb7322755 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:58:57.683587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:58:57.685319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:58:57.686684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 585/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 17:58:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:58:57.691171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:57.787020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3f4b-840a-4924-9db5-8a55770ca9d9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:57.809203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:57.821279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:57.821874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:57.822149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:57.822415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:57.823081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:57.823378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:57.824735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288f1e-8312-4681-b7b7-7e586be4684b None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:57.832031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 581/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Aug 9 17:58:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:59.829955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cef1e-7c6a-439f-adef-1af63eefd5d9 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:59.830513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cef1e-7c6a-439f-adef-1af63eefd5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:59.830513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cef1e-7c6a-439f-adef-1af63eefd5d9 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:59.831863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 586/1167] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:58:59.837034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7981f-96fb-4e88-967e-885e783978c2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:59.837455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7981f-96fb-4e88-967e-885e783978c2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:59.837563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7981f-96fb-4e88-967e-885e783978c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:59.855566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-72f7981f-96fb-4e88-967e-885e783978c2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:59.856759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-72f7981f-96fb-4e88-967e-885e783978c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:59.922720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 582/1168] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:58:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:58:59.945074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cef1e-7c6a-439f-adef-1af63eefd5d9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:00.043969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cef1e-7c6a-439f-adef-1af63eefd5d9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:00.077743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:00.078321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:00.078599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:00.078898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:00.079544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:00.080936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2382a31-87f3-45f1-a42c-aee2784c83af None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:00.227921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 587/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Tue Aug 9 17:58:59 2022] POST /identity/v3/domains => generated 267 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:59:02.225315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98135-e875-4cfe-b509-f302a743be41 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:02.225854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98135-e875-4cfe-b509-f302a743be41 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:02.225854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98135-e875-4cfe-b509-f302a743be41 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:02.227264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 583/1170] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:02.232615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:02.233046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:02.233136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:02.250851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:02.252033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:02.260972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:02.317629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 588/1171] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:59:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:59:02.341495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb98135-e875-4cfe-b509-f302a743be41 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:02.439720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb98135-e875-4cfe-b509-f302a743be41 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:02.455991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:02.476167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:02.476804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:02.477081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:02.477349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] PATH_INFO: `/v3/domains/3ce94c3622964755958ce62003caa9f2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:02.488275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] RBAC: Authorizing `identity:get_domain(domain_id=3ce94c3622964755958ce62003caa9f2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:02.489730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c77424-0776-4a03-ab03-d180c4f0059a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:02.491351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 584/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Tue Aug 9 17:59:02 2022] GET /identity/v3/domains/3ce94c3622964755958ce62003caa9f2 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:02.497492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:02.593184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f50d5-e5f6-4475-aedc-1f9c2c8a1356 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:02.625905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:02.626494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:02.626802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:02.627073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:02.627710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:02.629124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb8d57-1ea8-487e-849a-8988bbc405de None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:02.643840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:02.765955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 589/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 17:59:02 2022] POST /identity/v3/projects => generated 361 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:59:04.794035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04384-4b47-4eb0-8c35-03653c3fa2c5 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:04.794589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04384-4b47-4eb0-8c35-03653c3fa2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:04.794589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04384-4b47-4eb0-8c35-03653c3fa2c5 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:04.795920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 585/1174] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:04.801100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:04.801522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:04.801632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:04.819912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:04.821066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:04.884201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 590/1175] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:59:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:59:04.909386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac04384-4b47-4eb0-8c35-03653c3fa2c5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:05.008390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac04384-4b47-4eb0-8c35-03653c3fa2c5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:05.041298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:05.041903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:05.042181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:05.042458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] PATH_INFO: `/v3/domains/3ce94c3622964755958ce62003caa9f2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:05.043685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3ce94c3622964755958ce62003caa9f2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:05.045330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28d50b-3b7c-4fac-b4a6-2100d1f8b9d3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:05.046829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 586/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Tue Aug 9 17:59:04 2022] GET /identity/v3/domains/3ce94c3622964755958ce62003caa9f2 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:05.052441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:05.145866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff635-0381-4df7-ab6f-00a6af9a95fe None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:05.170421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:05.178479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:05.179107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:05.179383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:05.179645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:05.180265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:05.181674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9c5fd-c108-4232-9a71-3d01e732c29f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:05.271392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 591/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 17:59:05 2022] POST /identity/v3/users => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:59:07.320349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5387c-8680-4ee6-8956-34b96d227cae None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.320957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5387c-8680-4ee6-8956-34b96d227cae None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.320957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5387c-8680-4ee6-8956-34b96d227cae None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.322204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 587/1178] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:07.328022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.328428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.328581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.346888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:07.348063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:07.412276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 592/1179] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:59:07.434983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5387c-8680-4ee6-8956-34b96d227cae None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.481368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:07.536065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5387c-8680-4ee6-8956-34b96d227cae None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:07.567664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:07.571610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.571610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.571610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.579518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:59:07.584209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:59:07.585315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:59:07.586592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 588/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:59:07.592556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.688104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8f6f4-136f-4516-9ea9-9acb0e995308 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:07.701816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:07.721975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:07.722581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.722888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.723159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.723807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:07.724104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:07.725463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:07.733942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 593/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:07.739596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.836425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55050-6382-4abc-a538-5267c75a513e None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:07.838562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:07.869059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:07.869664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.869942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.870211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] PATH_INFO: `/v3/users/8036451f514146edb906fee5cd8e493a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.883790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] RBAC: Authorizing `identity:get_user(user_id=8036451f514146edb906fee5cd8e493a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:07.885221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:07.886841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 589/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:59:07 2022] GET /identity/v3/users/8036451f514146edb906fee5cd8e493a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:07.893692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.990291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df1e00-eed3-4bab-bc3b-81b88fd44ac7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.022523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.023151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.023424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.023690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.063819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:08.087884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.089425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.185018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 594/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:59:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:59:08.190896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.285927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84010-0216-4248-a8b0-d923395b1c4d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.318383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.319002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.319273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.319535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] PATH_INFO: `/v3/projects/785d70df2f3a4e16a3eb323d30995b4a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.320603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] RBAC: Authorizing `identity:get_project(project_id=785d70df2f3a4e16a3eb323d30995b4a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.322003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42190f6-13aa-4ab7-b024-6acb7824319c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.323590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 590/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:59:08 2022] GET /identity/v3/projects/785d70df2f3a4e16a3eb323d30995b4a => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:08.330072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.402170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:08.425541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738a9be-8108-4927-b6c9-8374badcee30 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.460700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.461300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.461579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.461846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.462563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=785d70df2f3a4e16a3eb323d30995b4a, user.id=8036451f514146edb906fee5cd8e493a) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:08.462896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.464318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021214e8-8ecd-46e6-968b-2f6542066b51 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.469920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 595/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:59:08 2022] GET /identity/v3/role_assignments?user.id=8036451f514146edb906fee5cd8e493a&scope.project.id=785d70df2f3a4e16a3eb323d30995b4a&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:10.461448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3315c-b18e-4ecd-8632-f7f448d2fa8b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:10.461998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3315c-b18e-4ecd-8632-f7f448d2fa8b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:10.461998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3315c-b18e-4ecd-8632-f7f448d2fa8b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:10.463341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 591/1186] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:10.469336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:10.469743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:10.469915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:10.487621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:10.488809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:10.552675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 596/1187] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:59:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:59:10.575162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3315c-b18e-4ecd-8632-f7f448d2fa8b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:10.605003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:10.674272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3315c-b18e-4ecd-8632-f7f448d2fa8b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:10.707080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:10.707681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:10.707956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:10.708222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:10.718102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:59:10.722602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:59:10.723734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 17:59:10.724938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 592/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:59:10.729683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:10.793727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:10.826615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d72b8-9c5c-4e56-9e2d-19c5a0b6a3bb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:10.858564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:10.859184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:10.859459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:10.859724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:10.860378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:10.860673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:10.862026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:10.869656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 597/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:10.876421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:10.963975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:10.985018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8bd-342d-40a2-b93b-8a50a0ad27bb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:11.006515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:11.007144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:11.007417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:11.007679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] PATH_INFO: `/v3/users/8036451f514146edb906fee5cd8e493a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:11.008769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] RBAC: Authorizing `identity:get_user(user_id=8036451f514146edb906fee5cd8e493a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:11.010171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:11.011731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 593/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:59:10 2022] GET /identity/v3/users/8036451f514146edb906fee5cd8e493a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:11.018036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:11.115295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e3d8-6003-4d86-a4f7-6c95733e870e None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:11.143500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:11.149084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:11.151389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:11.151389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:11.151389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] PATH_INFO: `/v3/projects/785d70df2f3a4e16a3eb323d30995b4a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:11.151389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] RBAC: Authorizing `identity:get_project(project_id=785d70df2f3a4e16a3eb323d30995b4a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:11.152892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a52b1-419e-455e-bfc9-86325625fa61 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:11.154521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 598/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:59:11 2022] GET /identity/v3/projects/785d70df2f3a4e16a3eb323d30995b4a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:11.160448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:11.255446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3dbb9-bad0-44f7-8479-3a633a761f1c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:11.288646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:11.289268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:11.289538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:11.289800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] PATH_INFO: `/v3/projects/785d70df2f3a4e16a3eb323d30995b4a/users/8036451f514146edb906fee5cd8e493a/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:11.291659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] RBAC: Authorizing `identity:create_grant(project_id=785d70df2f3a4e16a3eb323d30995b4a, user_id=8036451f514146edb906fee5cd8e493a, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:11.293148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba3a7-4391-4cbf-a3d7-4545ae9521fc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:11.357024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 594/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 17:59:11 2022] PUT /identity/v3/projects/785d70df2f3a4e16a3eb323d30995b4a/users/8036451f514146edb906fee5cd8e493a/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:13.336242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbccbc-a1d2-4034-95f0-dec444f0f337 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.336805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbccbc-a1d2-4034-95f0-dec444f0f337 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.336805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbccbc-a1d2-4034-95f0-dec444f0f337 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.338096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 599/1193] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 17:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:13.343378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.343809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.343904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.362061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:13.363242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:13.376567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:13.428200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 595/1194] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 17:59:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:59:13.451979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbccbc-a1d2-4034-95f0-dec444f0f337 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:13.551556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbccbc-a1d2-4034-95f0-dec444f0f337 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:13.584327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:13.584970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.585242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.585503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.596068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:59:13.600606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:59:13.607257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 17:59:13.608383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 600/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 17:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:59:13.609236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:13.703888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5887bbc-5ddf-483f-bffe-92d2b2061977 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:13.737279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:13.737879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.738154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.738430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.739108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:13.739433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:13.740793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:13.748633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 596/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 17:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:13.754399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:13.811975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:13.851970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9aec-cab5-46dd-9d18-fe90e7349cdc None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:13.885459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:13.886077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.886355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.887243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] PATH_INFO: `/v3/users/8036451f514146edb906fee5cd8e493a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.888427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] RBAC: Authorizing `identity:get_user(user_id=8036451f514146edb906fee5cd8e493a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:13.890133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:13.891733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 601/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Aug 9 17:59:13 2022] GET /identity/v3/users/8036451f514146edb906fee5cd8e493a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:13.897525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:13.910884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:13.995927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42f0bb-266a-47c3-a5fd-a8cb08fef220 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:14.028989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:14.029566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:14.029837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:14.030098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:14.094312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:14.095781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:14.106883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:14.190809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 597/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 17:59:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:59:14.196861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:14.292176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e0d58-2e47-49a4-9548-f3230da7fdf0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:14.316250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:14.325006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:14.325613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:14.325887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:14.330120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] PATH_INFO: `/v3/projects/785d70df2f3a4e16a3eb323d30995b4a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:14.330120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] RBAC: Authorizing `identity:get_project(project_id=785d70df2f3a4e16a3eb323d30995b4a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:14.330120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a75ca-2a91-490a-a855-4596528f44fd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:14.330595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 602/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Aug 9 17:59:14 2022] GET /identity/v3/projects/785d70df2f3a4e16a3eb323d30995b4a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:14.336308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:14.432346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a888-650c-43a1-a5dc-03cafc87e906 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:14.443723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:14.467716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:14.468311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:14.468583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:14.468843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:14.469550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=785d70df2f3a4e16a3eb323d30995b4a, user.id=8036451f514146edb906fee5cd8e493a) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:14.469848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:14.471295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc93b83-e16b-4b6b-9aeb-994634dd6b3a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:14.476883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 598/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 17:59:14 2022] GET /identity/v3/role_assignments?user.id=8036451f514146edb906fee5cd8e493a&scope.project.id=785d70df2f3a4e16a3eb323d30995b4a&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:59:24.345947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:24.474323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=13 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:25.411866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2863-0adf-4752-9cd0-83ba7c6907a1 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:25.412499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2863-0adf-4752-9cd0-83ba7c6907a1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:25.412499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2863-0adf-4752-9cd0-83ba7c6907a1 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:25.413723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 603/1201] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:25.419145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:25.419554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:25.419676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:25.438426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:25.439626 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:25.508231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 599/1202] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:00:25.534335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a2863-0adf-4752-9cd0-83ba7c6907a1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:25.634470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a2863-0adf-4752-9cd0-83ba7c6907a1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:25.668718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:25.669260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:25.669531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:25.669795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:25.680661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:25.682098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:25.696981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:25.698236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:25.699502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:25.699502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:25.699502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:25.699502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 604/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:25.704225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:25.803809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13040a05-332e-4834-b78a-17ceb6c61a2a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:25.833672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:25.839092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:25.839688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:25.839964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:25.840231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:25.840855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:25.841152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:25.842515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95929f6b-3010-49a4-88c9-0a9847bf3a0e None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:25.854354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 600/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:25.860109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:25.913432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:25.958920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde74b-7f8c-42dd-b148-bbe02ede5473 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:25.991224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:25.991810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:25.992079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:25.992337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:25.992965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:25.994491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc7c32-bdbe-44ce-b259-9f3b1de16f1a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:26.132969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 605/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Aug 9 18:00:25 2022] POST /identity/v3/users => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:00:28.197433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600a87-b3e0-4b54-a8c6-900aa0a26cff None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.197994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600a87-b3e0-4b54-a8c6-900aa0a26cff None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.197994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600a87-b3e0-4b54-a8c6-900aa0a26cff None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.199341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 601/1206] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:28.204621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.205366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.205366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.223314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:28.224506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:28.291299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 606/1207] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:00:28.314873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600a87-b3e0-4b54-a8c6-900aa0a26cff None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:28.416288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600a87-b3e0-4b54-a8c6-900aa0a26cff None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:28.449530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:28.450078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.450357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.450620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.469388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:28.474124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:28.475376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:28.476592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 602/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 18:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:28.481425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:28.522633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:28.580788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6a706-2c9d-485d-b330-72740f3453cd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:28.615134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:28.615733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.616006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.616273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.616923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:28.617219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:28.618577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:28.626244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 607/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 18:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:28.631996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:28.681319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:28.729676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403aaf6-d0f7-4483-b5c0-9826c75c7325 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:28.762104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:28.762746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.763022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.763287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.774565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:28.776040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:28.790023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:28.791194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:28.792280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:28.792280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:28.792280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:28.792280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 603/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:28.798096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:28.894884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:28.897766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87d550-ebe1-43e0-9cea-76145992cd48 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:28.932340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:28.932927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:28.933200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:28.933466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:28.934087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:28.934381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:28.935854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:28.946858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 608/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:28.952601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.052774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa463e1-723e-416f-be73-569f54abc97a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:29.054364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:29.085997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:29.086668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:29.086965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:29.087227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:29.095115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:29.096576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:29.107191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:29.108373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:29.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 604/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:29.114301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.213746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d9511-9539-4ea8-9d4c-a29efe2dbf83 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:29.245693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:29.246343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:29.246620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:29.246913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:29.247629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:29.247909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:29.249322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:29.267899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 609/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:29.273929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.376404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef88f9-0dc8-46f5-89d4-7559765abbd9 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:29.409537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:29.410130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:29.410405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:29.410672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:29.439316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:29.479127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:29.480585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:29.578988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 605/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 18:00:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:00:29.584859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.604788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:29.683540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e661-4f9f-4b0d-91bb-e5bb56fd4f2f None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:29.717051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:29.717665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:29.717938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:29.718203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:29.729388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:29.730902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:29.744792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:29.745923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:29.747115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:29.747115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:29.747115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:29.747115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 610/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:29.751992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.811483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:29.849336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cf2d-32e4-4185-be47-a809eb1747ba None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:29.882581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:29.883216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:29.883491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:29.883767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:29.884422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:29.884681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:29.886035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:29.897808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 606/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:29.903899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:29.974154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:30.004211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d9de-2321-47dd-9ba5-088a571e2319 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:30.038637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:30.039287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:30.039590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:30.039864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:30.046431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:30.048012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:30.058758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:30.059809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:30.060981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 611/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:30.066500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:30.165491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d1250-fa77-4e01-a447-ca7f693dfe91 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:30.187721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:30.199054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:30.199697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:30.199971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:30.200237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:30.200902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:30.201197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:30.202675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4aef9-c2f3-4f84-a830-b40d8f36c148 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:30.215329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 607/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:30.222045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:30.324190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933a44-dd42-493c-b4bf-e38a7d1a432d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:30.352347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:30.358054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:30.358638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:30.358982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:30.359359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:30.360150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=4ef94028a4914a688fd4a4539b1ade66) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:30.360453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:30.361860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0b583-cacf-4efc-b1e2-42dd49dbf9cf None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:30.367155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 612/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 18:00:30 2022] GET /identity/v3/role_assignments?user.id=4ef94028a4914a688fd4a4539b1ade66&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:32.343240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be932f-a677-492a-b3d8-0d275014e64c None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:32.344405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be932f-a677-492a-b3d8-0d275014e64c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:32.344405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be932f-a677-492a-b3d8-0d275014e64c None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:32.345688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 608/1220] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:32.351112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:32.351572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:32.351678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:32.369878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:32.371060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:32.435978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 613/1221] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:32.458396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be932f-a677-492a-b3d8-0d275014e64c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:32.559540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be932f-a677-492a-b3d8-0d275014e64c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:32.592666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:32.593422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:32.594579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:32.594579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:32.604120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:32.608702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:32.609714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:32.610810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 609/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 18:00:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:32.615926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:32.712467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f3d4-b49f-41b7-94f8-a7920784ff50 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:32.736994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:32.747809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:32.748414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:32.748733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:32.749007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:32.749678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:32.749967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:32.751375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:32.759150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 614/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 18:00:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:32.766032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:32.809132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:32.864728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d288b-3ca2-4e08-ba3a-8d7a97541ce8 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:32.899277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:32.899883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:32.900158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:32.900425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:32.911300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:32.912743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:32.926685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:32.927760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:32.928684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:32.928684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:32.928684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:32.928684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 610/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:32.933633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.032442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9883d-547b-4076-a0f3-0a37e00e3ff4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.065365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.065956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.066226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.066487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.067134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:33.067429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.068766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.080155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 615/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:33.085997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.179735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:33.186047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a65a84-af9b-43a0-86a6-7ca237836a24 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.218040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.219432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.219799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.220072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.227077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.228615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:33.239887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:33.240998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:33.242167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 611/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:33.246882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.292424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:33.345014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb42326-f706-4197-a9d9-8f23515683a7 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.377292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.377898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.378157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.378429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.379257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:33.379561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.380987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.399384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 616/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:33.405200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.504738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a693-c595-45c9-a3ef-d11534ee46b0 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.539438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.540056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.540334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.540592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.551547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.552982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:33.566955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:33.568103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:33.569235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:33.569235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:33.569235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:33.569235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 612/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:33.573813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.673913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b63e4-54f6-4c21-9882-132515fc646d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.676181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:33.706836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.707424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.707696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.707959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.708581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:33.708923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.710422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.721464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 617/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:33.727219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.736327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:33.823983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2e5f-53c6-4a5e-937d-3e09ccb8d7a7 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:33.856358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:33.856970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:33.857245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:33.857520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:33.865002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:33.866449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:33.877897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:33.887476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:33.888566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 613/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:33.889192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:33.986625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cc4f7-0dec-41c4-b180-78debb400d6c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:34.019629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:34.020213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:34.020488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:34.020756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:34.021429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:34.021731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:34.023182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d467bc-ae58-4f0c-b5b6-e05a1ff5169a None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:34.035018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 618/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:00:34.041117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:34.105915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:34.141215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e13c43-6edd-4264-881d-4449581e2a84 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:34.174219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:34.174878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:34.175150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:34.175414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/4ef94028a4914a688fd4a4539b1ade66/roles/666454a6407846b4936d1df7c782f605` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:34.191266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=4ef94028a4914a688fd4a4539b1ade66, role_id=666454a6407846b4936d1df7c782f605)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:34.192710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f926b87-7240-45ba-8574-3e479c59c5fb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:34.270432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 614/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 18:00:34 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/4ef94028a4914a688fd4a4539b1ade66/roles/666454a6407846b4936d1df7c782f605 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:00:36.285125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38047a0-8085-4b42-b5e5-49d6b64730d3 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:36.285731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38047a0-8085-4b42-b5e5-49d6b64730d3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:36.285731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38047a0-8085-4b42-b5e5-49d6b64730d3 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:36.287043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 619/1233] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:36.292260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422507b7-b62f-42dc-8854-73669f711474 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:36.292658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422507b7-b62f-42dc-8854-73669f711474 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:36.292805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422507b7-b62f-42dc-8854-73669f711474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:36.311927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-422507b7-b62f-42dc-8854-73669f711474 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:36.313127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-422507b7-b62f-42dc-8854-73669f711474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:36.378669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 615/1234] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:00:36.402303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38047a0-8085-4b42-b5e5-49d6b64730d3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:36.427552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:36.503169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38047a0-8085-4b42-b5e5-49d6b64730d3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:36.538178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:36.538789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:36.539067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:36.539334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:36.549513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:36.554245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:36.555431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:36.556270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 620/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Aug 9 18:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:36.561648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422507b7-b62f-42dc-8854-73669f711474 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:36.608070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:36.660805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422507b7-b62f-42dc-8854-73669f711474 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:36.694318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:36.694932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:36.695206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:36.695516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:36.696205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] RBAC: Adding query filter params (name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:36.696495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:36.697856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:36.706500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 616/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Aug 9 18:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:36.712282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:36.796071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:36.811527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42175d95-002d-47a3-b19a-56473827f6c0 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:36.846543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:36.847216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:36.847492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:36.847760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:36.857470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:36.858993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:36.873528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:36.875304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:36.876339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:36.876339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:36.876339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:36.876339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 621/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:36.881233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:36.981227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f2185-bee5-422d-887a-551380ef921a None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:36.990015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:37.014585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.015197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.016271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.016421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.017085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:37.017375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.018742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.030730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 617/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:37.036854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:37.138945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ba4f-34eb-45d5-8d66-86be9a9f1bd9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:37.170458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.171122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.171398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.172208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.174875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:37.179654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.181138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:37.192373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:37.193519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:37.194733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 622/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:37.199503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:37.297892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef2b46-60e6-401c-9bbf-1117e52e7355 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:37.331742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.332334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.332606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.332869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.333604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:37.333902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.335354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.352103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 618/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:37.358047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:37.456051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff008d-aba8-4201-b4da-fa2264b01ff9 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:37.491079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.491714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.491992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.492257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.555844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:37.562174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.563795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.658723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 623/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:00:37.664928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:37.691936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:37.761285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc252-4d3e-4817-aaab-3de2a8a49820 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:37.793951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.794586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.794902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.795170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.805583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.807063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:37.822095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:37.824920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:37.826122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:37.826122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:37.826122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:37.826122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 619/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:37.829918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:37.923013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:37.928601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d479721-7d80-403e-b71a-11964fd72b62 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:37.961924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:37.962509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:37.962806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:37.963073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:37.963691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:37.963983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:37.965329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:37.976712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 624/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:37.982775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:38.059516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:38.080095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99462-2a32-4a23-975f-3edf6b6510b3 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:38.112789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:38.113396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:38.113671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:38.113937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:38.120368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:38.121791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:38.132499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:38.133532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:38.134657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 620/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:38.139895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:38.241288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80097b-1591-42c0-9e89-f5793b49f214 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:38.276004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:38.276623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:38.276903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:38.277176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:38.277858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:38.278159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:38.279649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ed89f-2b53-4a7e-bc97-05c71a4d11e4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:38.291491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 625/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:38.297878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:38.397077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d04b-4c58-477f-9364-3cf4b7b227ed None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:38.429591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:38.430231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:38.430620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:38.430924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:38.435849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] RBAC: Adding query filter params (role.id=666454a6407846b4936d1df7c782f605, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=4ef94028a4914a688fd4a4539b1ade66) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:38.435849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:38.435849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227a18-1a0c-4a99-ac51-677333050f4f None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:38.440064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 621/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 18:00:38 2022] GET /identity/v3/role_assignments?user.id=4ef94028a4914a688fd4a4539b1ade66&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=666454a6407846b4936d1df7c782f605 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:40.554096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c72b0-272e-4e44-bbf8-4e21236ab7ae None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:40.554662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c72b0-272e-4e44-bbf8-4e21236ab7ae None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:40.554662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c72b0-272e-4e44-bbf8-4e21236ab7ae None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:40.555995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 626/1247] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:40.561297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:40.561715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:40.561856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:40.568817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:40.581027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:40.582192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:40.647051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 622/1248] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:00:40.670415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c72b0-272e-4e44-bbf8-4e21236ab7ae None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:40.689372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:40.770627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c72b0-272e-4e44-bbf8-4e21236ab7ae None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:40.805527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:40.806148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:40.806429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:40.806698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:40.816686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:40.821389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:40.822511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:40.823759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 627/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 18:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:40.829219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:40.930200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435902a-3637-4282-a641-0f389c5fcbcf None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:40.959637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:40.964465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:40.965173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:40.965453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:40.965724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:40.966380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:40.966680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:40.968076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:40.976570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 623/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 18:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:00:40.982458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:41.059353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:41.081606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb2748-8ea4-46c3-bf71-3792a3931e84 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:41.114813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:41.115435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:41.115720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:41.115993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:41.126737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:41.128201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:41.142803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:41.143900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:41.144782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:41.144782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:41.144782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:41.144782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 628/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:41.149991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:41.246555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358bf2c-a5d7-47c3-b1a8-433039385457 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:41.280676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:41.281435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:41.281668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:41.281932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:41.282556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:41.282874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:41.284231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:41.296088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 624/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:41.301967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:41.401274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a427a9e-2748-44c8-b376-3b60e1edf16d None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:41.426401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:41.434553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:41.435218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:41.435490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:41.435759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:41.443419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:41.444851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:41.456196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:41.457503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:41.458600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 629/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:41.469934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:41.524701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:41.569637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab227dc2-833f-4426-9c83-c55a53db56da None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:41.604175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:41.604774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:41.605050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:41.605318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:41.606177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:41.606479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:41.607938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:41.626069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 625/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:41.632500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:41.731305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350121a-b701-49ac-a502-c659ff0ba15b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:41.765296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:41.766086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:41.766363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:41.766650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:41.820903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:41.833034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:41.834511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:41.934261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 630/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 18:00:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:00:41.941077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:42.037472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de09c8-8da5-4223-bcb0-8b969f2e80fb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:42.050657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:42.070659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:42.071301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:42.071576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:42.071843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:42.081423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:42.083014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:42.097304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:42.099412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:42.100486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:42.100486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:42.100486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:42.100486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 626/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:42.104147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:42.192275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:42.204510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b111c-0ff0-4132-b0eb-68d60c882dbd None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:42.238677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:42.239303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:42.239580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:42.239846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:42.240474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:42.240924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:42.242760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:42.254626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 631/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:42.260537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:42.363142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d729-a953-45b5-afc3-f8b18ae60436 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:42.396594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:42.397314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:42.397590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:42.397885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:42.404386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:42.405819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:42.416843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:42.417766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:42.418760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 627/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:00:42.424672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:42.528019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55c54-0130-4b1a-a9ac-5e1a8393b033 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:42.561381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:42.562021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:42.562920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:42.563193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:42.563964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:42.564268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:42.565745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983c63-9603-43e7-be46-4ff6db189c4b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:42.578010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 632/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:42.584403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:42.610470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:42.686616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc530e16-e4ae-47b0-a405-98f60e0711fc None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:42.720776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:42.721359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:42.721630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:42.721894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:42.722596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=4ef94028a4914a688fd4a4539b1ade66) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:42.722920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:42.724344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fc15c-cef0-4a81-bbf1-eebd6efacafd None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:42.729603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 628/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 18:00:42 2022] GET /identity/v3/role_assignments?user.id=4ef94028a4914a688fd4a4539b1ade66&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:44.768283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de8927-d8fa-49d5-b1f8-feaf4db99e15 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:44.768855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de8927-d8fa-49d5-b1f8-feaf4db99e15 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:44.768855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de8927-d8fa-49d5-b1f8-feaf4db99e15 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:44.770165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 633/1261] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:44.775813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:44.776451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:44.776451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:44.799015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:44.800912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:44.882965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 629/1262] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:44.908546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de8927-d8fa-49d5-b1f8-feaf4db99e15 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:44.919451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.022277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de8927-d8fa-49d5-b1f8-feaf4db99e15 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.060240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.060876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.061156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.061430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.072007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:45.076716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:45.077755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:45.079100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 634/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 18:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:45.084329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:45.110296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.188965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aa43f-1e73-4d5b-942c-269d0d58346e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.223149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.223739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.224012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.224277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.224926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:45.225217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:45.226573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:45.234942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 630/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:45.240894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:45.312325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.345912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b82e5-ca54-4c2a-ab46-2409704a2d22 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.378577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.379225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.379502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.379771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.389676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:45.391665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:45.405975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:45.409678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:45.410836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:45.410836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:45.410836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:45.410836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 635/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:45.413597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:45.506121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.525590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f87724-2bbf-4ebd-ad3e-bcba3a3ae91c None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.558795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.559390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.559668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.559937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.560600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:45.560892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:45.562381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:45.574336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 631/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:45.580802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:45.680497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f096a-cd1a-48e0-9345-31624858925c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.700043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.714503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.715159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.715439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.715710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.723418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:45.724846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:45.735610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:45.739426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:45.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 636/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:45.742927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:45.842141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22de5f-4ef5-4b70-91fa-d991869c0286 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:45.874497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:45.875727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:45.876000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:45.876323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:45.877152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:45.877452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:45.878930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:45.887752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:45.896640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 632/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:45.902605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:46.011050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438723-743e-4dec-be1f-d3c35f8baf4c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:46.046044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:46.047760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:46.048052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:46.048322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:46.059542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:46.060978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:46.075669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:46.076916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:46.078005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:46.078005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:46.078005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:46.078005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 637/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:46.083166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:46.181309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03828ddd-8afa-4263-9e5a-f4fa783fd4cf None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:46.217045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:46.217677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:46.217952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:46.218221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:46.218904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:46.219204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:46.220701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:46.232102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 633/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:46.239402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:46.271803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:46.339508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3697ba-ece1-47ef-aa29-25e13581ad02 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:46.372518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:46.373131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:46.373450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:46.373720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:46.380724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:46.382136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:46.393096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:46.394042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:46.394996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 638/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:46.400351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:46.461451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:46.502058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dad23-d2d7-4efe-b8bd-6556339da2fe None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:46.534741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:46.535339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:46.535637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:46.535908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:46.536585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:46.536886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:46.538389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ff6d8-8976-44bd-8be5-3ab1c435a8ad None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:46.550790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 634/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:46.556869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:46.648606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:46.657657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5e2f-f0e7-4e2b-ad3d-458a16fd08b6 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:46.691540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:46.692222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:46.692500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:46.692772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] PATH_INFO: `/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/4ef94028a4914a688fd4a4539b1ade66/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:46.694647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dc6280ca54d8bb3b695ea7c93bb39, user_id=4ef94028a4914a688fd4a4539b1ade66, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:46.696164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ff164-ce43-44a0-8ae4-96667bf489a7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:46.777486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 639/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Aug 9 18:00:46 2022] PUT /identity/v3/projects/ef3dc6280ca54d8bb3b695ea7c93bb39/users/4ef94028a4914a688fd4a4539b1ade66/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:00:48.755837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590776-736e-4a02-ac4b-5f13ca63a0dc None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:48.756398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590776-736e-4a02-ac4b-5f13ca63a0dc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:48.756398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590776-736e-4a02-ac4b-5f13ca63a0dc None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:48.757693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 635/1274] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:48.762963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:48.763390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:48.767633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:48.781609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:48.783103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:48.848188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 640/1275] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:48.872019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590776-736e-4a02-ac4b-5f13ca63a0dc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:48.969227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590776-736e-4a02-ac4b-5f13ca63a0dc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:48.987141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:49.003707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.004317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.008401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.008401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.014786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:00:49.020145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:49.021269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:00:49.022384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:49.026284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 636/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Aug 9 18:00:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:49.027440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.106557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:49.124264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7156a03-2d57-40f0-91eb-c2b9f0b3719b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.158070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.158649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.158954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.159221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.159866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:49.160159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:49.161614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:49.173082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 641/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:00:49.178836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.279230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b99459-d3d7-472c-8e04-229b99595dcb None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.314246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.314894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.315168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.315432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.325066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:49.326493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:49.340957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:49.342071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:49.343213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:49.343213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:49.343213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:49.343213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 637/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:49.348477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.448681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f21d5a-f948-4dc9-84f7-ea8b9febb0b5 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.481768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.482361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.482637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.482932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.483558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:49.483853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:49.486142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:49.497400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 642/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:00:49.503442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.534748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:49.601888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b365e-1457-4f07-821a-85fed2ba13ff None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.635600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.636218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.636492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.636760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.645533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:49.647136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:00:49.658476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:49.662655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:49.663877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 638/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:49.665712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.669518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:49.764462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddad0e5-3ab4-42fd-94fd-8e7bd6298439 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.799784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.800305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.800576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.800838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:49.801533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:49.801828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:49.803269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:49.820851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 643/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:49.827818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:49.918153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:49.928115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab80fc-da80-4715-8f78-609d77460f25 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:49.960635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:49.961218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:49.961489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:49.961751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.027196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.028633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.114940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:50.124564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 639/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Aug 9 18:00:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:00:50.131196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:50.191914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:50.229108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75eaf-e9a5-4678-b238-f54ce0d4e5f4 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:50.262247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:50.263158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:50.263437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:50.263722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.273496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.275038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:00:50.290449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:50.291577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:00:50.292511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:00:50.292511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:00:50.292511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:50.292511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 644/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:50.298042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:50.393677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dad4e-ff25-49c1-b4c6-b0e5d677d1ea None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:50.426930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:50.427530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:50.427803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:50.428117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.428740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:50.429032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.430466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.441401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 640/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Aug 9 18:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:50.447206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:50.550350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04181d97-e5f9-4b75-b08a-b41bc7392e54 None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:50.564165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:50.584471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:50.585082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:50.585357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:50.585623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.592023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.594336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:00:50.605580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:50.606614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: ERROR keystone.server.flask.application Aug 09 18:00:50.611748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 645/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Aug 9 18:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:50.613814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:50.663528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:50.712653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c19606-f4fc-47a7-a540-e1c97f2117ef None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:50.747302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:50.749018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:50.749018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:50.749018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.749955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:50.750515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.752357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c9e29-4203-4ba4-a4a3-9874b4d9fac2 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.764935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 641/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Aug 9 18:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:50.771609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:50.874247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8bf0a-7f2c-4046-9e36-f512fd23b14b None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:50.908729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:50.909328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:50.909625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:50.909887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:50.910591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] RBAC: Adding query filter params (role.id=7409e93d755b445b891c0195b28879f6, scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39, user.id=4ef94028a4914a688fd4a4539b1ade66) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:50.910923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:50.917980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29ea50-49dd-49d9-a0ce-124ca5053106 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:50.918255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 646/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Aug 9 18:00:50 2022] GET /identity/v3/role_assignments?user.id=4ef94028a4914a688fd4a4539b1ade66&scope.project.id=ef3dc6280ca54d8bb3b695ea7c93bb39&role.id=7409e93d755b445b891c0195b28879f6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:52.974659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fbc93-b35f-4ad4-88fd-2226a6b2d5e8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:52.975295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fbc93-b35f-4ad4-88fd-2226a6b2d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:52.975295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fbc93-b35f-4ad4-88fd-2226a6b2d5e8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:52.976548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 642/1288] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:52.981549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:52.981995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:52.982174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:53.001414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:53.002584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:53.032747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:53.069631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 647/1289] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:53.091279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fbc93-b35f-4ad4-88fd-2226a6b2d5e8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:53.191764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fbc93-b35f-4ad4-88fd-2226a6b2d5e8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:53.202078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:53.224890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:53.225438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:53.225715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:53.225984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:53.226554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:53.227943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:53.236928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:53.238132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:53.239048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 643/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 18:00:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:53.244243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:53.340325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677d1f-5a48-47af-a678-8a2f06d71396 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:53.374381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:53.375180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:53.375612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:53.376041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:53.376615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:53.376918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:53.378339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0868e15-4d65-48a4-8495-15ee19dca479 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:53.383062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 648/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Aug 9 18:00:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:53.389266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:53.489173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8921-5535-4c5c-8edb-0ebb8c1ccbd4 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:53.522436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:53.523061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:53.523337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:53.523603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] PATH_INFO: `/v3/services` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:53.524280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] RBAC: Adding query filter params (type=placement) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:53.524523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:53.525884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de7c7d-07cd-4a4f-ac04-e190a7753cea None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:53.530250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 644/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Aug 9 18:00:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:55.543494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48439a-7c1b-4d70-90ed-f020180f503d None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:55.544036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48439a-7c1b-4d70-90ed-f020180f503d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:55.544132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48439a-7c1b-4d70-90ed-f020180f503d None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:55.545562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 649/1293] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:55.550997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4710-5b12-4103-b90d-7456e8aac567 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:55.551403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4710-5b12-4103-b90d-7456e8aac567 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:55.551545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4710-5b12-4103-b90d-7456e8aac567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:55.569739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-44bc4710-5b12-4103-b90d-7456e8aac567 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:55.570948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-44bc4710-5b12-4103-b90d-7456e8aac567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:55.600553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:55.636010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 645/1294] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:55.659918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48439a-7c1b-4d70-90ed-f020180f503d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:55.691045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:55.760536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48439a-7c1b-4d70-90ed-f020180f503d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:55.794982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:55.795573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:55.795845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:55.796109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:55.796915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:55.798826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e1167-627f-482e-a505-820f9109a1f6 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:55.871186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 650/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Aug 9 18:00:55 2022] POST /identity/v3/services => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:00:57.926053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fd31-9e39-4bf9-a8c5-1ef897c30339 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:57.926649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fd31-9e39-4bf9-a8c5-1ef897c30339 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:57.926649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fd31-9e39-4bf9-a8c5-1ef897c30339 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:57.928038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 646/1296] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:00:57.933108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:57.933537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:57.933628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:57.951341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:57.952504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:58.024299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 651/1297] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:00:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:00:58.048888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fd31-9e39-4bf9-a8c5-1ef897c30339 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:58.150604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fd31-9e39-4bf9-a8c5-1ef897c30339 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:58.184845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:58.185388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:58.185656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:58.185917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:58.186483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:58.188721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:00:58.197097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:00:58.198319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:00:58.199224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 647/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 18:00:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:00:58.205128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:58.223901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:58.302213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259ce00-2ec8-4e1f-a997-20fce328c851 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:58.335651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:58.336244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:58.336523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:58.336825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:58.337411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:58.337709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:58.339093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:58.345348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 652/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Aug 9 18:00:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:58.351356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:58.382003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:58.450236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f977-f97a-4057-b69d-33955e8f406d None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:58.484589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:58.485187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:58.485545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:58.485816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:58.486511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] RBAC: Adding query filter params (interface=public, service_id=dd3af89a38ce48b68adb6c2bb3d2f1ab, region_id=RegionOne) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:00:58.486847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:58.488213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586027e-a7ee-4a49-a7fc-bacc0da7e357 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:58.493101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 648/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Aug 9 18:00:58 2022] GET /identity/v3/endpoints?service_id=dd3af89a38ce48b68adb6c2bb3d2f1ab&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:00:58.498894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:00:58.581550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:58.599731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a8943-892c-4f87-836a-2816eddb891c None admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:00:58.633691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:00:58.634290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:58.634586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:58.634873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:58.635469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:00:58.636839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8358ef-a8ac-4526-a255-a9d7c244d840 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:00:58.643923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 653/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Aug 9 18:00:58 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:01:00.668233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6ff8-7d66-4be4-8a99-c69864252648 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:00.670803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6ff8-7d66-4be4-8a99-c69864252648 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:00.670803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6ff8-7d66-4be4-8a99-c69864252648 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:00.670803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 649/1302] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:01:00.676273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:00.676763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:00.676865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:00.694974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:01:00.696195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:00.761102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 654/1303] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:01:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:01:00.784770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec6ff8-7d66-4be4-8a99-c69864252648 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:00.878476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:00.885082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec6ff8-7d66-4be4-8a99-c69864252648 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:00.918315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:00.918954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:00.919231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:00.919498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:00.920068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:00.921426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:01:00.930419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:01:00.931508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application with Lock( Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: ERROR keystone.server.flask.application Aug 09 18:01:00.932345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 650/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Aug 9 18:01:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:01:00.938066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:01.011613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:01.034775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9293c-8a31-48ea-857f-0207680b8f15 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:01.068008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:01.068593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:01.068865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:01.069129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] PATH_INFO: `/v3/services` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:01.069764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:01:01.070059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:01.071584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59884327-6ef2-483e-a93b-d0325cfe7ca7 None admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:01.076511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 655/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Aug 9 18:01:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:01:01.082962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:01.182669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b40bec-7bc3-4fe6-84ce-d1a6e5222a29 None admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:01.215205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:01.215812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:01.216089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:01.216357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:01.216998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:01.218351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb53f6-2995-4950-bee2-fba09e1c8ce8 None admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:01.339468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 651/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:01:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:01:11.099709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=30 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:11.334940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:23.846829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2095-be91-45ed-8359-4274e804dd68 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:23.847985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2095-be91-45ed-8359-4274e804dd68 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:23.848474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2095-be91-45ed-8359-4274e804dd68 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:23.850548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 656/1307] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:23.855643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a778-8b9f-4176-b058-86f24b3d002b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:23.856101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a778-8b9f-4176-b058-86f24b3d002b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:23.856184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a778-8b9f-4176-b058-86f24b3d002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:23.937669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e223a778-8b9f-4176-b058-86f24b3d002b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:23.938912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e223a778-8b9f-4176-b058-86f24b3d002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:24.067132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 652/1308] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:24.106952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:24.107234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:24.107406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:24.109566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 657/1309] 10.222.0.36 () {54 vars in 965 bytes} [Tue Aug 9 18:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:24.115655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221e124-51aa-4e25-9b53-789020e63381 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:24.115949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221e124-51aa-4e25-9b53-789020e63381 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:24.116224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221e124-51aa-4e25-9b53-789020e63381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:24.281405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b221e124-51aa-4e25-9b53-789020e63381 None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:24.284397 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b221e124-51aa-4e25-9b53-789020e63381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:24.326874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:24.386873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 653/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Tue Aug 9 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:24.396442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:24.398343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:24.528634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b86a82-263e-4c4d-a17a-5275d4f4e16e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:24.613345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:24.613873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:24.614145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:24.614406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:24.616165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:24.681609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:24.755647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:24.757305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b0931-788b-48d0-8e6f-2dbafb32c236 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:24.779385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 658/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:27.125070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:27.125666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:27.125666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:27.127466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 654/1312] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:27.133478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804655-7956-4449-8a26-488ec22ac440 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:27.134026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804655-7956-4449-8a26-488ec22ac440 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:27.134026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804655-7956-4449-8a26-488ec22ac440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:27.227810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3a804655-7956-4449-8a26-488ec22ac440 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:27.229092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3a804655-7956-4449-8a26-488ec22ac440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:27.337107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 659/1313] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:27.376995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:27.378569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:27.494313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf7f3b-4e30-4220-8933-97b77894d66f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:27.570910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:27.573637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:27.574239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:27.574515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:27.574831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:27.577687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:27.692010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:27.693334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f2fec-6b85-4278-b4d7-c07b30b9f517 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:27.711985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 655/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:30.234196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:30.234804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:30.234804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:30.236169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 660/1315] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:30.241364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2393d8-c3fa-4ac9-a16e-4cbef1be80df None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:30.241755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2393d8-c3fa-4ac9-a16e-4cbef1be80df None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:30.241908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2393d8-c3fa-4ac9-a16e-4cbef1be80df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:30.306104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ea2393d8-c3fa-4ac9-a16e-4cbef1be80df None None] MFA Rules not processed for user `a5de9c855767447ba06e66c6b1bf1a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:30.307321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ea2393d8-c3fa-4ac9-a16e-4cbef1be80df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:30.358609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:30.402715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 656/1316] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:30.445055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:30.446576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:30.456572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:30.548726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a238b-b358-4e94-a968-28651ebf2800 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:30.618635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:30.619322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:30.619596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:30.619861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:30.621429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:30.682905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:30.727323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:30.728751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97926903-5776-4893-861b-f2d6d0a225b0 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:30.746508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 661/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:33.578568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:33.579160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:33.579160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:33.580496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 657/1318] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:33.585693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011d1d-f03e-479d-9542-78eb7684d289 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:33.586138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011d1d-f03e-479d-9542-78eb7684d289 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:33.586244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011d1d-f03e-479d-9542-78eb7684d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:33.652164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2f011d1d-f03e-479d-9542-78eb7684d289 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:33.653367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2f011d1d-f03e-479d-9542-78eb7684d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:33.749020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 662/1319] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:07:33.788075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:33.789687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:33.893490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4c05f-b745-462e-abcb-14b99e6d27f8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:33.965840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:33.966571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:33.966835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:33.967106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:33.968698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:33.976088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:34.080194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:34.081537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1f1b-7368-4143-8851-4b0b15d888f6 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:34.099205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 658/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:37.139477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914bce7-2975-426b-a501-3330fec173ba None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:37.140057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914bce7-2975-426b-a501-3330fec173ba None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:37.140151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914bce7-2975-426b-a501-3330fec173ba None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:37.141454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 663/1321] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:37.146922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc3e5b-2df8-485e-a540-9618335a92c0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:37.147352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc3e5b-2df8-485e-a540-9618335a92c0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:37.147468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc3e5b-2df8-485e-a540-9618335a92c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:37.211852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-f9cc3e5b-2df8-485e-a540-9618335a92c0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:37.213030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f9cc3e5b-2df8-485e-a540-9618335a92c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:37.269454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:37.308346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 659/1322] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:37.345124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914bce7-2975-426b-a501-3330fec173ba None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:37.346666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6914bce7-2975-426b-a501-3330fec173ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:37.366039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:37.464426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914bce7-2975-426b-a501-3330fec173ba None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:37.543847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:37.544454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:37.544782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:37.545089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:37.546927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:37.624265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:37.664029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:37.665463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954630f9-58f9-4918-bca0-3b8b75d72b8d service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:37.684495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 664/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:39.787079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd54-73a4-495c-a6a6-bd7394f362eb None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:39.787719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd54-73a4-495c-a6a6-bd7394f362eb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:39.787719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd54-73a4-495c-a6a6-bd7394f362eb None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:39.788947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 660/1324] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:39.794127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08528-06cf-44fa-867f-8625404dfb50 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:39.795150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08528-06cf-44fa-867f-8625404dfb50 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:39.795503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08528-06cf-44fa-867f-8625404dfb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:39.864382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c0e08528-06cf-44fa-867f-8625404dfb50 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:39.865499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c0e08528-06cf-44fa-867f-8625404dfb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:39.964649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 665/1325] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:40.019032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:40.019381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:40.019576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:40.020950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 661/1326] 10.222.0.36 () {54 vars in 965 bytes} [Tue Aug 9 18:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:07:40.026949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb610-b2da-41e6-933c-01a5708334bc None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:40.027300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb610-b2da-41e6-933c-01a5708334bc None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:40.027633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdb610-b2da-41e6-933c-01a5708334bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:40.059065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:40.104524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d7bdb610-b2da-41e6-933c-01a5708334bc None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:40.105710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d7bdb610-b2da-41e6-933c-01a5708334bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:40.180499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 666/1327] 10.222.0.36 () {58 vars in 1062 bytes} [Tue Aug 9 18:07:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:40.189742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:40.191279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:40.301129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c5ea4-3157-444e-ac48-675bfa001032 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:40.375129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:40.375645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:40.375916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:40.376182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:40.377747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:40.394968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:40.501281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:40.502794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84230514-3c34-42bd-b370-aaded1c0b508 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:40.521392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 662/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:42.812995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:42.813534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:42.813534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:42.815080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 667/1329] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:42.819971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5552e3-d124-4b19-8192-aa152159379c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:42.820401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5552e3-d124-4b19-8192-aa152159379c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:42.820499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5552e3-d124-4b19-8192-aa152159379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:42.887961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7f5552e3-d124-4b19-8192-aa152159379c None None] MFA Rules not processed for user `a5de9c855767447ba06e66c6b1bf1a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:42.889730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7f5552e3-d124-4b19-8192-aa152159379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:42.956113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:42.989338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 663/1330] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:43.027331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:43.029040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:43.148633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ddb55-c353-407e-9a4e-211abe513944 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:43.173251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:43.229305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:43.229916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:43.230193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:43.230489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:43.232825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:43.342813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:43.344333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b358b2a-7bcc-437d-9fbc-bfc44dede6ca service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:43.362559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 668/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:46.190617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:46.191227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:46.191227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:46.192551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 664/1332] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:46.197860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6ba3-6dc8-439b-8571-8165b498bd75 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:46.198257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6ba3-6dc8-439b-8571-8165b498bd75 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:46.198407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6ba3-6dc8-439b-8571-8165b498bd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:46.261468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:46.264223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e61d6ba3-6dc8-439b-8571-8165b498bd75 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:46.265410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e61d6ba3-6dc8-439b-8571-8165b498bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:46.359487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 669/1333] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:46.395494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:46.397049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:46.500064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6245654-16cb-40c8-a6c5-f788c2fa3c52 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:46.571461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:46.572053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:46.572331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:46.572603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:46.574193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:46.600970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:46.682310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:46.683776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0c69-1c00-4f66-92e8-6aebb9645e48 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:46.702315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 665/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:48.738370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:48.738971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:48.738971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:48.740278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 670/1335] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:48.745550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0302-6151-4b42-a6fa-27298d5427d2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:48.746007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0302-6151-4b42-a6fa-27298d5427d2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:48.746166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0302-6151-4b42-a6fa-27298d5427d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:48.812946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3b0a0302-6151-4b42-a6fa-27298d5427d2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:48.814150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3b0a0302-6151-4b42-a6fa-27298d5427d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:48.895407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:48.917700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 666/1336] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:48.959404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:48.960923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:49.064579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ce5b-f5b4-479f-8d8d-932918a9071f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:49.093010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:49.137509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:49.138694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:49.138994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:49.139262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:49.140858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:49.247940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:49.249373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062fe84-a036-46bc-b6bb-f1ca10b2ad95 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:49.268341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 671/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:52.003926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:52.004502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:52.004502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:52.005792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 667/1338] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:52.011136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27642fc-edbd-483e-81e6-40817a59ae93 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:52.011636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27642fc-edbd-483e-81e6-40817a59ae93 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:52.011735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27642fc-edbd-483e-81e6-40817a59ae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:52.075693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e27642fc-edbd-483e-81e6-40817a59ae93 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:52.076886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e27642fc-edbd-483e-81e6-40817a59ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:52.084964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:52.172075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 672/1339] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:07:52.209799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:52.211356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:52.314544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7f667-3208-44fd-8b89-565fff19a840 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:52.386005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:52.386590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:52.386890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:52.387170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:52.388751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:52.423415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:52.495665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:52.497182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10278012-7669-4406-9d6d-3731d47348d6 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:52.515546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 668/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:56.964207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:56.964843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:56.964843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:56.967099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 673/1341] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:07:56.972335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1581-4656-4cc3-89e8-57ecb96ecb23 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:56.972796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1581-4656-4cc3-89e8-57ecb96ecb23 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:56.972890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1581-4656-4cc3-89e8-57ecb96ecb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:57.042129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e79a1581-4656-4cc3-89e8-57ecb96ecb23 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:07:57.043341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e79a1581-4656-4cc3-89e8-57ecb96ecb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:57.127858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:57.139865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 669/1342] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:07:57.180110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:57.181629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:57.287008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a644218-f1bb-468d-8bc3-24c5bf90e4fa None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:57.325704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:57.361038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:57.361643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:57.361917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:57.362184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:57.363784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:07:57.477066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:57.478503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cce029-5019-432e-bab8-c47f492fa918 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:57.500095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 674/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:08:00.489274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:00.489832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:00.489832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:00.491210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 670/1344] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:00.496394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6a57-6c29-43c8-9d73-3446d77cb94d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:00.496867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6a57-6c29-43c8-9d73-3446d77cb94d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:00.497131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6a57-6c29-43c8-9d73-3446d77cb94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:00.562251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-944e6a57-6c29-43c8-9d73-3446d77cb94d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:00.563497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-944e6a57-6c29-43c8-9d73-3446d77cb94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:00.581381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:00.660015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 675/1345] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:00.698068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:00.700300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:00.804868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaf4aa-331d-403b-b941-78512bb8148d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:00.880312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:00.880911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:00.881188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:00.881455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:00.883321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:00.926669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:00.993627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:00.995109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef280d62-c4d9-4eef-a663-ab23955f920a service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:01.014553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 671/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:05.349146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:05.349673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:05.349766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:05.351088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 676/1347] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:05.356551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2717e-2935-4595-ad6e-b7e654fd5a68 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:05.357042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2717e-2935-4595-ad6e-b7e654fd5a68 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:05.357158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2717e-2935-4595-ad6e-b7e654fd5a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:05.427233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3ab2717e-2935-4595-ad6e-b7e654fd5a68 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:05.428457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3ab2717e-2935-4595-ad6e-b7e654fd5a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:05.537734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:05.540650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 672/1348] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:05.580144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:05.581770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:05.695514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a781-2dc2-4e7b-8195-010e243796d2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:05.749080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:05.779675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:05.780481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:05.780867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:05.781352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:05.783379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:05.900780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:05.902246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43527e1-d1b3-46f8-8128-c6ec440673f1 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:05.923463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 677/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:10.924221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ac93-609a-499f-add5-dff483306890 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:10.924997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ac93-609a-499f-add5-dff483306890 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:10.925432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ac93-609a-499f-add5-dff483306890 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:10.927200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 673/1350] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:10.932631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762780-a5aa-450d-a923-074de8239b60 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:10.933375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762780-a5aa-450d-a923-074de8239b60 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:10.933819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762780-a5aa-450d-a923-074de8239b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:11.076539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c9762780-a5aa-450d-a923-074de8239b60 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:11.077730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c9762780-a5aa-450d-a923-074de8239b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:11.099642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:11.224993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 678/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:08:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:08:15.476910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f4e4-e863-4131-a9fd-aa98ed2afc04 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:15.477801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f4e4-e863-4131-a9fd-aa98ed2afc04 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:15.478138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f4e4-e863-4131-a9fd-aa98ed2afc04 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:15.479669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 674/1352] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:15.484958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379527f-bb1f-4df2-92c7-aa3eb3913714 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:15.485416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379527f-bb1f-4df2-92c7-aa3eb3913714 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:15.485513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379527f-bb1f-4df2-92c7-aa3eb3913714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:15.573609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7379527f-bb1f-4df2-92c7-aa3eb3913714 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:15.574912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7379527f-bb1f-4df2-92c7-aa3eb3913714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:15.654824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:15.691983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 679/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:08:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:16.879026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a348d63-af63-412d-9f41-0c7f231bbf1e None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:16.879554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a348d63-af63-412d-9f41-0c7f231bbf1e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:16.879554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a348d63-af63-412d-9f41-0c7f231bbf1e None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:16.880876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 675/1354] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:16.886615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163b71-d1eb-46d2-b61b-c9b9dd66b0a0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:16.888755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163b71-d1eb-46d2-b61b-c9b9dd66b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:16.888755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163b71-d1eb-46d2-b61b-c9b9dd66b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:16.965592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a4163b71-d1eb-46d2-b61b-c9b9dd66b0a0 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:16.966894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a4163b71-d1eb-46d2-b61b-c9b9dd66b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:17.073305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 680/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:20.541715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a703e-400e-4e88-9690-ee6c752a4e9d None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:20.542615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a703e-400e-4e88-9690-ee6c752a4e9d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:20.543213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a703e-400e-4e88-9690-ee6c752a4e9d None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:20.545337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 676/1356] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:20.551107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cb7b-5277-41b8-8d2d-a7cbae627266 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:20.551920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cb7b-5277-41b8-8d2d-a7cbae627266 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:20.552193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cb7b-5277-41b8-8d2d-a7cbae627266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:20.606354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:20.641991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-af94cb7b-5277-41b8-8d2d-a7cbae627266 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:20.643618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-af94cb7b-5277-41b8-8d2d-a7cbae627266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:20.749971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 681/1357] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:20.808773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:20.809060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:20.809340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:20.811235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 677/1358] 10.222.0.36 () {54 vars in 962 bytes} [Tue Aug 9 18:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:20.822609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b583b-24ee-4fe4-9185-be5a071ca8b3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:20.823260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b583b-24ee-4fe4-9185-be5a071ca8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:20.823752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b583b-24ee-4fe4-9185-be5a071ca8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:20.921041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-570b583b-24ee-4fe4-9185-be5a071ca8b3 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:20.922257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-570b583b-24ee-4fe4-9185-be5a071ca8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:20.976959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:21.050253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 682/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Aug 9 18:08:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:08:21.059647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:21.061407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:21.214132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b600984-865a-4a97-89ad-769c693bc826 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:21.295614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:21.340594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:21.341257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:21.341507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:21.341823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:21.343464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:21.442486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:21.444126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8478b2-3c96-4783-994c-c80e348a4539 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:21.469123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 678/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:23.756674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:23.757227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:23.757227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:23.758582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 683/1361] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:23.765062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1db526-964b-4c05-91cd-aa160e490ace None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:23.765568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1db526-964b-4c05-91cd-aa160e490ace None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:23.765669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1db526-964b-4c05-91cd-aa160e490ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:23.837922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1a1db526-964b-4c05-91cd-aa160e490ace None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:23.839119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1a1db526-964b-4c05-91cd-aa160e490ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:23.882583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:23.947898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 679/1362] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:24.000142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:24.001974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:24.173973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22089ca-72f1-4e85-8c5a-c34c1c5fb4ba None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:24.184931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:24.305314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:24.307279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:24.307279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:24.307279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:24.317007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:24.419169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:24.420936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caab855-e8d7-496f-8e9c-5e7a4f388670 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:24.440381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 684/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:26.951097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565c15-32ea-4ee7-b04e-032be00b94ea None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:26.951637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565c15-32ea-4ee7-b04e-032be00b94ea None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:26.951637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565c15-32ea-4ee7-b04e-032be00b94ea None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:26.953179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 680/1364] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:26.958570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bfed-4c2a-4d66-a446-9ad574478e11 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:26.959149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bfed-4c2a-4d66-a446-9ad574478e11 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:26.959149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bfed-4c2a-4d66-a446-9ad574478e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:26.976684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:27.039751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a782bfed-4c2a-4d66-a446-9ad574478e11 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:27.040949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a782bfed-4c2a-4d66-a446-9ad574478e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.151114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 685/1365] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:27.194218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:27.194502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:27.194808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:27.196253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 681/1366] 10.222.0.36 () {54 vars in 962 bytes} [Tue Aug 9 18:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:27.203194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa255814-daee-406b-9cc0-75f0041f2964 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:27.203643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa255814-daee-406b-9cc0-75f0041f2964 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:27.203984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa255814-daee-406b-9cc0-75f0041f2964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:27.287267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-fa255814-daee-406b-9cc0-75f0041f2964 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:27.288801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fa255814-daee-406b-9cc0-75f0041f2964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.331745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:27.416566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 686/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Aug 9 18:08:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:27.425256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:27.426894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.599341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0c2d5-bca8-4730-98b2-393b605982ca None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:27.622406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:27.719018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:27.719524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:27.719796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:27.720064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:27.721647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.816201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:27.817970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050bfb5-8017-4c3d-9674-060572cef99b service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:27.837913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 682/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:28.897422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e98a-175d-44cf-9128-609c34db0e1b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:28.898018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e98a-175d-44cf-9128-609c34db0e1b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:28.898018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e98a-175d-44cf-9128-609c34db0e1b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:28.899531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 687/1369] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:28.904896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235cfe-40f2-4d0c-8f0a-6cd944e46ad8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:28.905323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235cfe-40f2-4d0c-8f0a-6cd944e46ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:28.905491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235cfe-40f2-4d0c-8f0a-6cd944e46ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:28.938913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:28.973636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4c235cfe-40f2-4d0c-8f0a-6cd944e46ad8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:28.975824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4c235cfe-40f2-4d0c-8f0a-6cd944e46ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.076106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 683/1370] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:29.422993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:29.422993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:29.422993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:29.425303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 688/1371] 10.222.0.36 () {54 vars in 964 bytes} [Tue Aug 9 18:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:29.428988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc400-f7f0-42a6-949d-715dcc165f6a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:29.429346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc400-f7f0-42a6-949d-715dcc165f6a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:29.429700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc400-f7f0-42a6-949d-715dcc165f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:29.529786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5bbbc400-f7f0-42a6-949d-715dcc165f6a None None] MFA Rules not processed for user `886cb33a3f254d20b25936c39b3410d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:29.531020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5bbbc400-f7f0-42a6-949d-715dcc165f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.579604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:29.610366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 684/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Tue Aug 9 18:08:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:08:29.618241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:29.619939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.734073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6899dc1-defd-45c9-88f8-5071d141ab7e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:29.779183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:29.816638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:29.817485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:29.818598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:29.819455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:29.821517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.952031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:29.953271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78693a2-5f9f-41ba-a9ff-1bf393854c22 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:29.972340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 689/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:30.127449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:30.127932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:30.128322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:30.129930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 685/1374] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:30.135215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76599f-5c06-4585-8905-a3a28b99e191 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:30.135543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76599f-5c06-4585-8905-a3a28b99e191 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:30.135846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76599f-5c06-4585-8905-a3a28b99e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:30.220054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4b76599f-5c06-4585-8905-a3a28b99e191 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:30.221555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4b76599f-5c06-4585-8905-a3a28b99e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:30.246961 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:30.330082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 690/1375] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:30.378554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:30.380324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:30.533616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dc721-a092-4cad-b7a2-8bfb27e262fa None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:30.584447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:30.647959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:30.648556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:30.648832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:30.653910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:30.653910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:30.739222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:30.740808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08685d-a73e-445b-803c-d8dc30450edd service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:30.760471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 686/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:32.963284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:32.963835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:32.963956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:32.966292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 691/1377] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:32.971685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7de878-2281-415a-86c3-5143d4581eb5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:32.972100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7de878-2281-415a-86c3-5143d4581eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:32.972230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7de878-2281-415a-86c3-5143d4581eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:33.047207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bc7de878-2281-415a-86c3-5143d4581eb5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:33.048473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bc7de878-2281-415a-86c3-5143d4581eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:33.055750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:33.162645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 687/1378] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:33.208280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:33.209777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:33.355870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b369f9e-8cf3-4dc7-88d6-4423dff44620 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:33.359546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:33.488324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:33.488324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:33.488324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:33.488324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:33.489473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:33.632803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:33.635288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c292a35-884c-4907-9a0f-4746c737cf67 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:33.670952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 692/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:35.860753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:35.861325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:35.861325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:35.862826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 688/1380] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:35.867949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39441-2980-4f66-9e72-0f9718a1eb0e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:35.868393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39441-2980-4f66-9e72-0f9718a1eb0e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:35.868774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39441-2980-4f66-9e72-0f9718a1eb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:35.880864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:35.941955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-63c39441-2980-4f66-9e72-0f9718a1eb0e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:35.944485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-63c39441-2980-4f66-9e72-0f9718a1eb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:36.056628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 693/1381] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:36.101744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:36.103510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:36.248372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:36.259920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0e973-465e-436d-af3e-af10a9683373 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:36.378388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:36.379417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:36.379756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:36.380089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:36.381970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:36.487460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:36.489497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a30bd-9c66-4057-b0cd-6bb7b5bdf4fe service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:36.503929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:36.514957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 689/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:38.908976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:38.909536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:38.909536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:38.910887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 694/1383] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:38.916467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724265-6712-4b84-bb72-cca9ddaf9252 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:38.916960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724265-6712-4b84-bb72-cca9ddaf9252 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:38.917911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724265-6712-4b84-bb72-cca9ddaf9252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:38.989451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-44724265-6712-4b84-bb72-cca9ddaf9252 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:38.990682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-44724265-6712-4b84-bb72-cca9ddaf9252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:39.095898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 690/1384] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:39.140183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:39.142213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:39.239453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:39.304132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ed0c6-3d6a-40e7-b247-6c370b6fc235 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:39.431895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:39.432494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:39.433147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:39.433147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:39.434764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:39.496883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:39.540429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:39.542669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1187634-c94e-4ed1-a61a-dca20a99dba2 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:39.563718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 695/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:41.674783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:41.676001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:41.676001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:41.677614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 691/1386] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:41.683100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351150-d985-4510-9d7d-fbf35dfebf7b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:41.683538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351150-d985-4510-9d7d-fbf35dfebf7b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:41.683722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351150-d985-4510-9d7d-fbf35dfebf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:41.756839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a0351150-d985-4510-9d7d-fbf35dfebf7b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:41.758181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a0351150-d985-4510-9d7d-fbf35dfebf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:41.875725 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 696/1387] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:41.922640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:41.924189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:42.004211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:42.095590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0271d3-7b26-47b8-8863-fa2865809ae2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:42.213444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:42.214107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:42.214450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:42.214639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:42.216264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:42.255155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:42.312050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:42.314043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121bad-c8a8-48ec-9ab3-98e6ef2a8cf1 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:42.335005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 692/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:44.555645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:44.556337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:44.556337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:44.557662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 697/1389] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:08:44.562942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6abdd-7c70-4690-9b71-d72348e24188 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:44.563408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6abdd-7c70-4690-9b71-d72348e24188 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:44.563519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6abdd-7c70-4690-9b71-d72348e24188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:44.628873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-65e6abdd-7c70-4690-9b71-d72348e24188 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:44.630054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-65e6abdd-7c70-4690-9b71-d72348e24188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:44.726967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 693/1390] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:44.771510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:44.773070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:44.804514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:44.920615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d8a95-6379-460d-9be2-b48e3c335f23 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:45.021580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:45.033087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:45.033700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:45.033972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:45.034236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:45.035969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:45.125863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:45.127650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2c1f4-6b7d-43ce-ba08-fb068458ec7d service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:45.149334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 698/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:45.430896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77367c9-b0a4-4bcd-9fe9-c17d125c9299 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:45.431319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77367c9-b0a4-4bcd-9fe9-c17d125c9299 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:45.431590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77367c9-b0a4-4bcd-9fe9-c17d125c9299 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:45.433553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 694/1392] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:45.439337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f013-9792-4df3-9974-763f8871b5e2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:45.439622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f013-9792-4df3-9974-763f8871b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:45.439894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f013-9792-4df3-9974-763f8871b5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:45.515984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-99a3f013-9792-4df3-9974-763f8871b5e2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:45.517191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99a3f013-9792-4df3-9974-763f8871b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:45.576218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:45.616398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 699/1393] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:45.674543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:45.675017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:45.675125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:45.676465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 695/1394] 10.222.0.36 () {54 vars in 967 bytes} [Tue Aug 9 18:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:45.681415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0778-a6e8-4ab2-88c2-25fefe5af2d1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:45.681703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0778-a6e8-4ab2-88c2-25fefe5af2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:45.681982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0778-a6e8-4ab2-88c2-25fefe5af2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:45.761122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e4cd0778-a6e8-4ab2-88c2-25fefe5af2d1 None None] MFA Rules not processed for user `f814ce7767d645c58a5c00a9672c54eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:45.762322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e4cd0778-a6e8-4ab2-88c2-25fefe5af2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:45.837123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 700/1395] 10.222.0.36 () {58 vars in 1064 bytes} [Tue Aug 9 18:08:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:45.845252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:45.846763 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:45.861322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:45.952596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555762-ebdb-4111-a5fb-a1916a0ba2a8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:46.025739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:46.027075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:46.027327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:46.027594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:46.029177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:46.096800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:46.139128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:46.140579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2ad09-5e71-40b0-8b74-8d4fd85eb822 service glance] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:46.159029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 696/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:47.086338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254314-28f3-4171-ad26-76b6162a8f3a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.086963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254314-28f3-4171-ad26-76b6162a8f3a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.086963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254314-28f3-4171-ad26-76b6162a8f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.157172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4f254314-28f3-4171-ad26-76b6162a8f3a None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:47.158433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4f254314-28f3-4171-ad26-76b6162a8f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.168089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:47.250904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 701/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:47.298517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54a3d-d08e-4261-aa0f-91372f214c04 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.299059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54a3d-d08e-4261-aa0f-91372f214c04 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.299203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54a3d-d08e-4261-aa0f-91372f214c04 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.300548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 697/1398] 10.222.0.36 () {54 vars in 943 bytes} [Tue Aug 9 18:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:47.307581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.308599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.308960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.328355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89cd6d-551d-43fb-9eaa-34524b82d012 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.328567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89cd6d-551d-43fb-9eaa-34524b82d012 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.328834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89cd6d-551d-43fb-9eaa-34524b82d012 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.330187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 698/1399] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:08:47.335029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.335303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.335569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.392703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] MFA Rules not processed for user `2c61ea9321f94455afe35b931b9fcf8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:47.393901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.402435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:47.403660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.482892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:47.506739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 702/1400] 10.222.0.36 () {58 vars in 1040 bytes} [Tue Aug 9 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:47.518507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 699/1401] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:08:47.518666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:47.527990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf26d-c2ad-42e6-a36d-13a8cc1e9b93 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:47.563136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:47.564826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.601257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:47.647361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:47.648485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.648757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.649025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.651211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.702543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4ae2-46cf-43c6-9ed0-1eb7462f9ab7 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:47.745418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:47.747165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528c046-f7a7-4408-844f-4237298d1ac1 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:47.756941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 703/1402] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:08:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:08:47.839685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:47.840319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:47.840592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:47.840853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:47.842464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:47.862950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:47.932954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:47.934600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5fab6-6e00-4e9a-be53-2d176c9c61df service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:47.952516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 700/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:08:50.228162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:50.228763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:50.228889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:50.230643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 704/1404] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:50.236475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059c9a-34a8-45e2-986a-f8fa434b299d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:50.236907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059c9a-34a8-45e2-986a-f8fa434b299d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:50.237208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059c9a-34a8-45e2-986a-f8fa434b299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:50.320359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-11059c9a-34a8-45e2-986a-f8fa434b299d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:50.321896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-11059c9a-34a8-45e2-986a-f8fa434b299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:50.423189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:50.429193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 701/1405] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:50.483354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:50.484934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:50.494956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:50.659257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483649c-79be-463b-bdee-7278a5c2b108 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:50.828295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:50.829010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:50.829313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:50.829585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:50.831770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:51.060686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:51.062307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b79443-82e8-490b-a00b-4a3f04882e20 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:51.081966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 705/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:51.296013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:53.505338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:53.505827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:53.505923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:53.508473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 702/1407] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:08:53.513067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc994b4-b843-4835-8bbe-55e7c5af01e6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:53.513488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc994b4-b843-4835-8bbe-55e7c5af01e6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:53.513660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc994b4-b843-4835-8bbe-55e7c5af01e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:53.586662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-1cc994b4-b843-4835-8bbe-55e7c5af01e6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:53.587927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1cc994b4-b843-4835-8bbe-55e7c5af01e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:53.641588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:53.695176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 706/1408] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:53.741615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:53.743158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:53.907589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213e985-1a79-4753-9112-d8b82bad3c86 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:53.984513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:54.035679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:54.036283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:54.036561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:54.036830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:54.038433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:54.152670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:54.154275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139f206-6654-43a9-a7a5-aaec52084694 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:54.178961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 703/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:03.693309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=23 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:04.174867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=64 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:11.863395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d64f7-5060-41ec-80a6-21168f0969ed None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:11.863987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d64f7-5060-41ec-80a6-21168f0969ed None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:11.863987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d64f7-5060-41ec-80a6-21168f0969ed None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:11.865292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 707/1410] 10.222.0.36 () {54 vars in 918 bytes} [Tue Aug 9 18:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:09:11.871181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55562-194b-4879-a2a1-a29c49705cc4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:11.871619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55562-194b-4879-a2a1-a29c49705cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:11.871767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55562-194b-4879-a2a1-a29c49705cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:11.946554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-17b55562-194b-4879-a2a1-a29c49705cc4 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:11.948368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-17b55562-194b-4879-a2a1-a29c49705cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:12.052284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 704/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Aug 9 18:09:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:12.067419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15616036-c7c4-4608-a4fe-36267af3687f None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:12.067701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15616036-c7c4-4608-a4fe-36267af3687f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:12.067975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15616036-c7c4-4608-a4fe-36267af3687f None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:12.069284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 708/1412] 10.222.0.36 () {54 vars in 918 bytes} [Tue Aug 9 18:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:12.074562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3c42-a197-4d53-a319-86605738c44d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:12.074875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3c42-a197-4d53-a319-86605738c44d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:12.075199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3c42-a197-4d53-a319-86605738c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:12.146243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:12.149927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-251c3c42-a197-4d53-a319-86605738c44d None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:12.151166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-251c3c42-a197-4d53-a319-86605738c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:12.266926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 705/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Aug 9 18:09:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:14.739934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:14.740454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:14.740618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:14.741946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:14.748807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7374f5-f156-438e-aa39-3a0caeb42b20 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:14.749261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7374f5-f156-438e-aa39-3a0caeb42b20 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:14.749357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7374f5-f156-438e-aa39-3a0caeb42b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:14.814319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4b7374f5-f156-438e-aa39-3a0caeb42b20 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:14.815554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4b7374f5-f156-438e-aa39-3a0caeb42b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:14.870781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:14.912005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:14.955812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:14.957333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:15.099719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b5d44-6101-4733-bed8-32e2644b5b2b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:15.171080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:15.210789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:15.211385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:15.211655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:15.211916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:15.214647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:15.304981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:15.307344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfffe05-cf89-4afd-909f-3135e8eb705e service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:15.324883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 710/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:16.224158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:16.224713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:16.224713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:16.226111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:16.231216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab668fc-6a1d-4f1d-b28c-0227758456cc None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:16.231653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab668fc-6a1d-4f1d-b28c-0227758456cc None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:16.231787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab668fc-6a1d-4f1d-b28c-0227758456cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:16.307454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4ab668fc-6a1d-4f1d-b28c-0227758456cc None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:16.308643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4ab668fc-6a1d-4f1d-b28c-0227758456cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:16.335059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:16.405951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:16.560211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:16.561714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:16.706971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355094b8-ead2-4354-a6a2-0c2a2fc5bd77 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:16.747119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:16.822611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:16.823156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:16.823432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:16.823700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:16.825300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:16.914630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:16.916312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c701e-86eb-4f42-987b-49beb4dbcf26 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:16.934078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 708/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:17.824374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:17.824972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:17.824972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:17.826253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:17.831233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833936a-359e-46c1-ba28-abf3f9a6e634 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:17.831644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833936a-359e-46c1-ba28-abf3f9a6e634 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:17.831757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833936a-359e-46c1-ba28-abf3f9a6e634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:17.885737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:17.907004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3833936a-359e-46c1-ba28-abf3f9a6e634 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:17.908251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3833936a-359e-46c1-ba28-abf3f9a6e634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:18.003404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:18.155524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:18.157018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:18.301009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1090b-be35-4831-be7c-420f735f134b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:18.313749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:18.412839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:18.413346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:18.413614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:18.413876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:18.416004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:18.505031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:18.506600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e268-3ca9-4571-9a01-5c988416e6d6 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:18.525317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 713/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:19.393694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:19.394248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:19.394248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:19.395634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:19.400790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40a46-86fa-447d-8f18-5c0e9620ec6c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:19.401229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40a46-86fa-447d-8f18-5c0e9620ec6c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:19.401420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40a46-86fa-447d-8f18-5c0e9620ec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:19.420685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:19.475880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-56c40a46-86fa-447d-8f18-5c0e9620ec6c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:19.477108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-56c40a46-86fa-447d-8f18-5c0e9620ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:19.573513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:19.724393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:19.725848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:19.836614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:19.874043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb4f7d-88d9-4137-a11a-0ab0c298fa21 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:19.987274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:19.987870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:19.988141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:19.988401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:19.989965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:20.065679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:20.078410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:20.080029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba8fea-f48c-4115-a6be-25e720ac651c service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:20.098327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 711/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:20.968620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:20.969162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:20.969162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:20.970396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:20.975486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcdd9f-9e46-42ec-9849-a228beb98618 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:20.975928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcdd9f-9e46-42ec-9849-a228beb98618 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:20.976038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcdd9f-9e46-42ec-9849-a228beb98618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:21.050356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e0bcdd9f-9e46-42ec-9849-a228beb98618 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:21.051587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e0bcdd9f-9e46-42ec-9849-a228beb98618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:21.147000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:21.299864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:21.301370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:21.387503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:21.450445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61780-3f27-4daf-95d5-ea28871c8c11 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:21.565245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:21.565765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:21.566034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:21.566294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:21.567884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:21.614971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:21.657587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:21.659181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12049fb-a2b1-43b1-9c5d-f3678661d886 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:21.677212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 716/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:22.550477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11816-7465-40f1-9378-157c09088015 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:22.551091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11816-7465-40f1-9378-157c09088015 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:22.551091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11816-7465-40f1-9378-157c09088015 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:22.552539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:22.557968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178a85-f86f-44e3-8913-4fd510806b17 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:22.558418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178a85-f86f-44e3-8913-4fd510806b17 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:22.558513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178a85-f86f-44e3-8913-4fd510806b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:22.633832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4e178a85-f86f-44e3-8913-4fd510806b17 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:22.635895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4e178a85-f86f-44e3-8913-4fd510806b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:22.730925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:22.884455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11816-7465-40f1-9378-157c09088015 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:22.885967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-86b11816-7465-40f1-9378-157c09088015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:22.934342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:23.033159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11816-7465-40f1-9378-157c09088015 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:23.148277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:23.148878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:23.149154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:23.150010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:23.151794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:23.158597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:23.241344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:23.242974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618337-b1be-4a62-a03c-86995e3d05e9 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:23.261525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 714/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:24.118889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:24.119449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:24.119449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:24.120769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:24.125833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f71b8d-c3da-47b2-8ac6-c4de3030bedc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:24.130822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f71b8d-c3da-47b2-8ac6-c4de3030bedc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:24.130822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f71b8d-c3da-47b2-8ac6-c4de3030bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:24.204087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-33f71b8d-c3da-47b2-8ac6-c4de3030bedc None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:24.205300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-33f71b8d-c3da-47b2-8ac6-c4de3030bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:24.271755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:24.300643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:24.451902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:24.453470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:24.472223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:24.598840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53397-254d-4953-aa0f-6ec1e3cd75f0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:24.684080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:24.709126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:24.709742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:24.710015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:24.710282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:24.712040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:24.800915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:24.802501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddd99f-8547-4cf7-a4d5-54aa1022c6e7 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:24.820583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 719/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:25.702308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:25.702847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:25.702976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:25.704231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:25.709241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8522-d951-4530-97fc-194f14009885 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:25.709681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8522-d951-4530-97fc-194f14009885 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:25.709785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8522-d951-4530-97fc-194f14009885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:25.787433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ee4e8522-d951-4530-97fc-194f14009885 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:25.788697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ee4e8522-d951-4530-97fc-194f14009885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:25.830830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:25.888750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:26.040111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:26.041589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:26.186581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8d9c-4086-4d39-9f78-4f7e4b65e30b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:26.239272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:26.297807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:26.298334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:26.298614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:26.298980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:26.300573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:26.390041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:26.391662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6472376-a8e3-40a1-a0ee-391dc1b50e46 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:26.409377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 717/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:27.321471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:27.322044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:27.322044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:27.323377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:27.328299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacfb1-803c-4b23-8552-c94f3ad7304b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:27.328708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacfb1-803c-4b23-8552-c94f3ad7304b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:27.328841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacfb1-803c-4b23-8552-c94f3ad7304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:27.404056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-02cacfb1-803c-4b23-8552-c94f3ad7304b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:27.405163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-02cacfb1-803c-4b23-8552-c94f3ad7304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:27.407200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:27.503324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:27.654850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:27.656285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:27.800040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d9d6a-3a55-4cee-9ebc-f5785e79e1bf None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:27.827708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:27.913651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:27.914221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:27.914544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:27.915505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:27.917217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:28.006022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:28.007621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001b042-8866-4370-946c-d13e29aaeff2 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:28.026861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 722/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:29.945858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:29.946848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:29.946848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:29.947762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:29.953004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29c1c-19dd-41b4-a96e-5bc5813d8824 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:29.953437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29c1c-19dd-41b4-a96e-5bc5813d8824 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:29.953533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29c1c-19dd-41b4-a96e-5bc5813d8824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:29.994108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:30.018432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d6e29c1c-19dd-41b4-a96e-5bc5813d8824 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:30.019691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d6e29c1c-19dd-41b4-a96e-5bc5813d8824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:30.114686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:30.155250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:30.156785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:30.259370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2fc7b-ceab-45de-ae45-d7263e829344 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:30.291368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:30.332098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:30.332688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:30.332958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:30.333218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:30.334848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:30.445339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:30.447029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6302bd-f9b0-4fdf-9678-c62224efbb91 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:30.588726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 720/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:32.755060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:32.755664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:32.755664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:32.757012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:32.762310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5674-7230-4a51-bc10-2786dbffb2e9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:32.762771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5674-7230-4a51-bc10-2786dbffb2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:32.762966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5674-7230-4a51-bc10-2786dbffb2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:32.827861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-dc9d5674-7230-4a51-bc10-2786dbffb2e9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:32.829057 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-dc9d5674-7230-4a51-bc10-2786dbffb2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:32.842182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:32.926211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:32.964152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:32.965668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:33.065814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7e08-b62f-4544-976a-0b77cb3316a3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:33.077439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:33.138299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:33.138919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:33.139191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:33.139456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:33.141014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:33.246524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:33.247987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f05eed-e3c4-456e-828a-62d840ebe331 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:33.268045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 725/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:35.620268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:35.620832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:35.620832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:35.622138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:35.627295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052769-a1e5-450c-ae19-2c4f047eacdd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:35.627745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052769-a1e5-450c-ae19-2c4f047eacdd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:35.627853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052769-a1e5-450c-ae19-2c4f047eacdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:35.675705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:35.693172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-66052769-a1e5-450c-ae19-2c4f047eacdd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:35.694392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-66052769-a1e5-450c-ae19-2c4f047eacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:35.790974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:35.830383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:35.831962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:35.935001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d1384-0a6f-4d1b-8633-6960f18c6ee7 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:35.975847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:36.011087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:36.011682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:36.011952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:36.012213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:36.013790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:36.121605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:36.122981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f36612-f056-4dd0-b5cc-51da9a11acc1 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:36.141482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 723/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:38.262931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:38.263481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:38.263481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:38.264817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:38.270230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860445-08b2-452c-8d2b-e9cac52cca73 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:38.270639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860445-08b2-452c-8d2b-e9cac52cca73 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:38.270806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860445-08b2-452c-8d2b-e9cac52cca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:38.334347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-29860445-08b2-452c-8d2b-e9cac52cca73 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:38.335561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-29860445-08b2-452c-8d2b-e9cac52cca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:38.356236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:38.432577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:38.471938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:38.473503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:38.576981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665d162-8050-47c8-b812-c76f0e3e80f2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:38.596175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:38.649524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:38.650114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:38.650390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:38.650658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:38.652259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:38.761552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:38.763021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e1126-b120-4634-8b27-62fe4ba85a06 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:38.781368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 728/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:40.829931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:40.830514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:40.830514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:40.831957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:40.837270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef35575-f6bb-49ec-be85-f38c70853e81 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:40.837687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef35575-f6bb-49ec-be85-f38c70853e81 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:40.837780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef35575-f6bb-49ec-be85-f38c70853e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:40.900998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:40.903561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4ef35575-f6bb-49ec-be85-f38c70853e81 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:40.904747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4ef35575-f6bb-49ec-be85-f38c70853e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:41.000377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:41.038263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:41.040355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:41.142404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65211e-a1f3-465d-a37a-28b80322a08e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:41.190882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:41.214859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:41.215454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:41.215726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:41.215993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:41.217558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:41.326751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:41.328409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8580-a44d-43aa-baf1-9c331f816229 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:41.347086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 726/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:43.408207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:43.408790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:43.408790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:43.410070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:43.415307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07cf3-79c5-47a5-9133-78ddac023931 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:43.415716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07cf3-79c5-47a5-9133-78ddac023931 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:43.415842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07cf3-79c5-47a5-9133-78ddac023931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:43.481230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a0a07cf3-79c5-47a5-9133-78ddac023931 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:43.482411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a0a07cf3-79c5-47a5-9133-78ddac023931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:43.512345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:43.578755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:43.617966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:43.619387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:43.723710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d17ae0-2d12-4163-ae98-1c1f94bad173 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:43.752119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:43.797158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:43.797754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:43.798028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:43.798332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:43.799957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:43.908000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:43.909471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f80b8-de52-4483-986c-b8694bf94002 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:43.928423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 731/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:46.008840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:46.009383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:46.009383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:46.010721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:46.016063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27e92-4281-441b-9e43-9d973c3c2f55 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:46.016503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27e92-4281-441b-9e43-9d973c3c2f55 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:46.016608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27e92-4281-441b-9e43-9d973c3c2f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:46.080912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-41b27e92-4281-441b-9e43-9d973c3c2f55 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:46.082099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-41b27e92-4281-441b-9e43-9d973c3c2f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:46.091100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:46.179609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:09:46.218258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:46.219860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:46.322213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf9ec2-1bce-451e-aabe-c9f578c4019b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:46.375859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:46.394744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:46.395342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:46.395617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:46.395908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:46.397500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:46.504958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:46.506398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e99e62-c2e2-449e-93cc-e96e7b0b9cf7 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:46.525088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 729/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:48.660410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:48.661019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:48.661019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:48.662435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:48.667693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396fac5-6225-4c3d-817b-22b2e470eb47 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:48.668103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396fac5-6225-4c3d-817b-22b2e470eb47 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:48.668239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396fac5-6225-4c3d-817b-22b2e470eb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:48.734292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e396fac5-6225-4c3d-817b-22b2e470eb47 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:48.735514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e396fac5-6225-4c3d-817b-22b2e470eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:48.774698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:48.832616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:48.870641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:48.872203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:48.973500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0bdd6-d062-4ce5-ad63-8081f8259990 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:49.011702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:49.049748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:49.050353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:49.050629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:49.050925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:49.052514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:49.162846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:49.164841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346802-c901-44e7-a685-01316bedd59d service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:49.185441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 734/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:51.286088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:51.286722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:51.286722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:51.288027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:51.294001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed4f5-91dd-42f6-9372-f49d0a01f2e5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:51.294453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed4f5-91dd-42f6-9372-f49d0a01f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:51.294566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed4f5-91dd-42f6-9372-f49d0a01f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:51.359751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-05eed4f5-91dd-42f6-9372-f49d0a01f2e5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:51.360948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-05eed4f5-91dd-42f6-9372-f49d0a01f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:51.379971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:51.462736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:51.500518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:51.502081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:51.609090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34781f29-270e-423c-ad26-f3815a4dba57 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:51.674256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:51.681058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:51.681655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:51.681928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:51.682195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:51.683873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:51.792590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:51.794012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4baaf-958b-4547-8023-c8e41bf02355 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:51.811686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 732/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:53.861082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:53.861672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:53.861672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:53.862999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:09:53.869149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9f3e-2be1-42ff-ae02-8696a44ae41a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:53.869566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9f3e-2be1-42ff-ae02-8696a44ae41a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:53.869683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9f3e-2be1-42ff-ae02-8696a44ae41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:53.934827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1c2c9f3e-2be1-42ff-ae02-8696a44ae41a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:09:53.936217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1c2c9f3e-2be1-42ff-ae02-8696a44ae41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:53.983328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:54.033273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:09:54.071129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:54.072682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:54.174108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5dcdb-7c04-4514-b12c-78e83b41329b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:54.221019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:54.247834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:54.248467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:54.248978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:54.249271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:54.250892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:54.358630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:54.360085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ce8f1-4e92-40c5-9609-880a3f82c999 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:54.379679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 737/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:04.031509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=22 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:04.377780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=66 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:14.143751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ab565-fef4-456e-b2c9-a8f42a8919cd None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.144375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ab565-fef4-456e-b2c9-a8f42a8919cd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.144375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ab565-fef4-456e-b2c9-a8f42a8919cd None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.145601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 734/1471] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:14.151246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d68cd-440b-4601-b3f7-9f651fdf1475 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.151688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d68cd-440b-4601-b3f7-9f651fdf1475 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.151803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d68cd-440b-4601-b3f7-9f651fdf1475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.220531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7f0d68cd-440b-4601-b3f7-9f651fdf1475 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:14.221846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7f0d68cd-440b-4601-b3f7-9f651fdf1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:14.320129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 738/1472] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:14.404884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.405162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.405427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.406772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 735/1473] 10.222.0.36 () {54 vars in 967 bytes} [Tue Aug 9 18:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:14.411807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc4a4-41c8-4868-a56a-acc0eccbaba7 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.412092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc4a4-41c8-4868-a56a-acc0eccbaba7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.412440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc4a4-41c8-4868-a56a-acc0eccbaba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.488793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-14abc4a4-41c8-4868-a56a-acc0eccbaba7 None None] MFA Rules not processed for user `f814ce7767d645c58a5c00a9672c54eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:14.489987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-14abc4a4-41c8-4868-a56a-acc0eccbaba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:14.510550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:14.565424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 739/1474] 10.222.0.36 () {58 vars in 1064 bytes} [Tue Aug 9 18:10:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:14.573301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:14.575331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:14.678674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f06b1-c81f-46f6-88c3-be7684184ee0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:14.750363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:14.750985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.751257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.751520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.753353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:14.807868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:14.864035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:14.865657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989380-c979-485f-ad0b-ac8af7c41fd3 service glance] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:14.884922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 736/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:17.003978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:17.004558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:17.004558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:17.005922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 740/1476] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:17.011263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08ef8-8a10-40d3-b731-5154ef7e4fac None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:17.011721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08ef8-8a10-40d3-b731-5154ef7e4fac None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:17.011828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08ef8-8a10-40d3-b731-5154ef7e4fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:17.076089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-70a08ef8-8a10-40d3-b731-5154ef7e4fac None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:17.077276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-70a08ef8-8a10-40d3-b731-5154ef7e4fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:17.173531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 737/1477] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:17.198371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:17.199960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:17.325324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b3965-bf25-44f1-9c92-16d5b57be2a1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:17.372989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:17.418248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:17.418882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:17.419161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:17.419430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:17.420060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:17.422508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:10:17.422508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:10:17.422508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:10:17.423840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c03585-0404-4dcc-8c81-73078736e1a4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:17.442937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 741/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Aug 9 18:10:17 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:10:19.476721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:19.477230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:19.477333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:19.478592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 738/1479] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:10:19.483765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fa1c-b3c1-420e-b94d-7e0b4b5e83d2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:19.484179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fa1c-b3c1-420e-b94d-7e0b4b5e83d2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:19.484324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fa1c-b3c1-420e-b94d-7e0b4b5e83d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:19.550652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7028fa1c-b3c1-420e-b94d-7e0b4b5e83d2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:19.551874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7028fa1c-b3c1-420e-b94d-7e0b4b5e83d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:19.648842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 742/1480] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:19.691743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:19.693793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:19.703484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:19.838755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9139ca-8bc2-41df-9f31-fb715b09479e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:19.917631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:19.950985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:19.951579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:19.951850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:19.952114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:19.953690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:20.041134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:20.042742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78f05c-245e-4b2d-866b-2dfb3aa5039c service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:20.060069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 739/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:10:22.246481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:22.247092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:22.247092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:22.248394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 743/1482] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:22.253802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde51-2762-47b1-abe5-a9bb3747514c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:22.254263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde51-2762-47b1-abe5-a9bb3747514c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:22.254371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde51-2762-47b1-abe5-a9bb3747514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:22.319241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a93bde51-2762-47b1-abe5-a9bb3747514c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:22.320434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a93bde51-2762-47b1-abe5-a9bb3747514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:22.365461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:22.415698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 740/1483] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:22.453799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:22.455394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:22.478797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:22.558926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10252-9ad9-40cf-aae5-d02bea4c3a1b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:22.633235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:22.633833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:22.634104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:22.634376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:22.636003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:22.710477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:22.744919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:22.746365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a119a-07da-4bb5-9b81-6a4aa2246751 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:22.764298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 744/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:24.848187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:24.848758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:24.848758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:24.850839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 741/1485] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:24.856137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda1f-eb61-4c14-9d88-e882e502d808 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:24.856553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda1f-eb61-4c14-9d88-e882e502d808 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:24.856689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda1f-eb61-4c14-9d88-e882e502d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:24.922374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9a9cda1f-eb61-4c14-9d88-e882e502d808 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:24.926303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9a9cda1f-eb61-4c14-9d88-e882e502d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:25.019669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 745/1486] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:25.058830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:25.060354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:25.164256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da37a9c-4a22-4a3a-b12b-2472602d4c58 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:25.234467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:25.236349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:25.236947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:25.237226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:25.237522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:25.239180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:25.347657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:25.349221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5097f9-f903-495e-b65c-4c9b834fcc34 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:25.367407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 742/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:27.464040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:27.464623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:27.464623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:27.466062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 746/1488] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:27.471029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a28b8-8192-4a4f-9e2c-79dc0f47b945 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:27.471497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a28b8-8192-4a4f-9e2c-79dc0f47b945 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:27.471610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a28b8-8192-4a4f-9e2c-79dc0f47b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:27.536178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-639a28b8-8192-4a4f-9e2c-79dc0f47b945 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:27.537493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-639a28b8-8192-4a4f-9e2c-79dc0f47b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:27.589659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:27.633090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 743/1489] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:27.672602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:27.674159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:27.711153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:27.777548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d60a3-d834-479a-a738-99dd29c7aa47 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:27.849595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:27.850204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:27.850478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:27.850775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:27.852369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:27.930501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:27.958894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:27.960320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ed749-14c6-4c6e-9ee7-7d599d41a299 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:27.978289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 747/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:30.351946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:30.352535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:30.352535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:30.353848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 744/1491] 10.222.0.36 () {54 vars in 927 bytes} [Tue Aug 9 18:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:10:30.359012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3bb0-deba-4dd7-afcb-2e242172ad60 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:30.359456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3bb0-deba-4dd7-afcb-2e242172ad60 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:30.359565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3bb0-deba-4dd7-afcb-2e242172ad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:30.426133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-335c3bb0-deba-4dd7-afcb-2e242172ad60 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:30.427542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-335c3bb0-deba-4dd7-afcb-2e242172ad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:30.525578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 748/1492] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Aug 9 18:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:30.567289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:30.568335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:30.671707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437f9bd-dbd8-4ac8-87f2-2f3ae2d0c449 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:30.744463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:30.745053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:30.745325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:30.745582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:30.747211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:30.758211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:30.857499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:30.858969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a300e-cf68-4873-8ee3-e85b8a846a81 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:30.878037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 745/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:40.523754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=82 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:40.875468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=50 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:50.617411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:50.618008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:50.618008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:50.693135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:11:50.694330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:50.789585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 749/1494] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:11:50.798306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:50.798890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:50.798890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:50.874236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:11:50.875629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:50.971434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 746/1495] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:11:50.976218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:50.977459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:50.988857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e23e9-bd4f-41fd-983d-32f7addeffee None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:51.049818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:51.082593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:51.083218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:51.083488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:51.083753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:51.084378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:11:51.084669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:51.085700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:51.085700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:11:51.085700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:11:51.086582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:51.098300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 750/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:11:51.106778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:51.115761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a106c-74d4-4aaf-bc08-d25b18e26ec3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:51.177164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:51.211130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:51.211729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:51.212002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:51.212269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:51.212897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:51.214624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:51.214624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:11:51.214624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:11:51.215840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:51.335556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 747/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:11:51 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:11:51.339727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:51.341328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:51.375792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e0ebb-10f7-401e-bbb2-c31e4ca11fc1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:51.470494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:51.471125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:51.471400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:51.471665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:51.472292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:51.473286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:51.473286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:11:51.473286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:11:51.474488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:51.597890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 751/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:11:51 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:11:51.602194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:51.603702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:51.665627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7702-aa0a-40c8-8bfb-9fac5b38bd4f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:51.746208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:51.757159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:51.757753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:51.758024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:51.758284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:51.758940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:51.759930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:51.759930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:11:51.759930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:11:51.760857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:51.773157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 748/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:11:51.777282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:51.778938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:51.790584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78fbef-3ad2-4f19-a0de-9eca8e081d0a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:51.816870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:51.887659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:51.888185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:51.888464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:51.888734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:51.889367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:51.890366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:51.890366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:11:51.890366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:11:51.891312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cde6e-0236-4153-891c-9cd7a9f929eb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:51.903640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 752/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:11:51.908003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:51.909545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:51.942823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81485ba2-e622-417f-b5e3-9c9fe2c0079d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:52.036543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:52.037189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.037460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.037723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] PATH_INFO: `/v3/projects/f520e018c6fc47f18372b6b70e45a6a4/users/8fc65809147f42f184d3eca667fc971e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.059385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f520e018c6fc47f18372b6b70e45a6a4, user_id=8fc65809147f42f184d3eca667fc971e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:52.060408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:52.060408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:11:52.060408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:11:52.062345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be37a5-18a6-410c-b6d5-5e29599b1cc7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:52.140247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 749/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:11:51 2022] PUT /identity/v3/projects/f520e018c6fc47f18372b6b70e45a6a4/users/8fc65809147f42f184d3eca667fc971e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:11:52.152405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fedb08-8aa1-4cc0-8e8b-6a99eba48b23 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.156994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fedb08-8aa1-4cc0-8e8b-6a99eba48b23 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.156994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fedb08-8aa1-4cc0-8e8b-6a99eba48b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.222917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d2fedb08-8aa1-4cc0-8e8b-6a99eba48b23 None None] MFA Rules not processed for user `8fc65809147f42f184d3eca667fc971e`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:11:52.224117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d2fedb08-8aa1-4cc0-8e8b-6a99eba48b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:52.305485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 753/1502] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:11:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:11:52.318898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.319182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.319449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.320772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 750/1503] 10.222.0.36 () {54 vars in 964 bytes} [Tue Aug 9 18:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:11:52.325619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf1682-09cc-40e7-b463-847ef131b2b0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.325907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf1682-09cc-40e7-b463-847ef131b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.326179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf1682-09cc-40e7-b463-847ef131b2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.357306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:52.401504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2baf1682-09cc-40e7-b463-847ef131b2b0 None None] MFA Rules not processed for user `886cb33a3f254d20b25936c39b3410d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:11:52.402626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2baf1682-09cc-40e7-b463-847ef131b2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:52.478731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 754/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Tue Aug 9 18:11:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:11:52.486551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:52.488084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:52.537585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:52.590354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb31321-51f5-451b-980b-81316cadc0a2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:52.666103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:52.666624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.666943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.667210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.668804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:52.768364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:52.769839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:52.773323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:52.787820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 751/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:11:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:11:52.833753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3a6b-1ad1-4e73-bcfe-229a77cffd42 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:52.834201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3a6b-1ad1-4e73-bcfe-229a77cffd42 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:52.834336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3a6b-1ad1-4e73-bcfe-229a77cffd42 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:52.835610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 755/1506] 10.222.0.36 () {52 vars in 830 bytes} [Tue Aug 9 18:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:11:52.839067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:52.840271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:52.935791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e890d80-b8c5-467d-8c04-3bd674e457cb service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:53.030167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:53.030818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:53.031095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:53.032001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] PATH_INFO: `/v3/users/8fc65809147f42f184d3eca667fc971e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:53.044078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc65809147f42f184d3eca667fc971e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:53.045084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:53.045084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:11:53.045084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:11:53.046287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:53.056538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:53.263429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc65809147f42f184d3eca667fc971e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:53.277259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc65809147f42f184d3eca667fc971e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:53.300435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc65809147f42f184d3eca667fc971e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:53.367252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc65809147f42f184d3eca667fc971e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:53.367769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc65809147f42f184d3eca667fc971e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:53.374192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 752/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:11:52 2022] DELETE /identity/v3/users/8fc65809147f42f184d3eca667fc971e => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:11:53.379402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:53.379687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:53.379963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:53.436621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:11:53.438475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:53.513971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:53.526057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 756/1508] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:11:53.536308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:53.537570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:53.639406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a1382-e57b-4fa3-85b8-f7887f60b7d1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:53.693079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:53.712224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:53.712827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:53.713101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:53.713367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:53.715054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:53.854545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:53.855942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f5fa-59b4-4411-8959-3e6f7b7afb40 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:53.874311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 753/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:11:54.353811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:54.355108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:54.367964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93ce2-a952-4261-a8b8-d6c8a0bf09cb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:54.462269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:54.462930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:54.463211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:54.463481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] PATH_INFO: `/v3/projects/f520e018c6fc47f18372b6b70e45a6a4` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:54.474289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f520e018c6fc47f18372b6b70e45a6a4)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:54.475370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:11:54.475370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:11:54.475370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:11:54.476607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:54.589823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f520e018c6fc47f18372b6b70e45a6a4', 'request_id': 'req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:54.595894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f520e018c6fc47f18372b6b70e45a6a4', 'request_id': 'req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:54.608862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f520e018c6fc47f18372b6b70e45a6a4', 'request_id': 'req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:54.710530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] The token cache is being invalidate because project f520e018c6fc47f18372b6b70e45a6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:11:54.710787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:54.711184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1398cc3c-eb4f-4f4d-bd0c-fd6452992474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:11:54.712587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 757/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:11:54 2022] DELETE /identity/v3/projects/f520e018c6fc47f18372b6b70e45a6a4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:12:03.871822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=62 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:12:04.708519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:24.021297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85855-3020-4149-bec0-801890386d5f None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:24.021923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85855-3020-4149-bec0-801890386d5f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:24.021923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85855-3020-4149-bec0-801890386d5f None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:24.023228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:24.028945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2b785-c6c7-4acd-806f-aa03ab2eeb56 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:24.029393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2b785-c6c7-4acd-806f-aa03ab2eeb56 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:24.029502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2b785-c6c7-4acd-806f-aa03ab2eeb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:24.109673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0dc2b785-c6c7-4acd-806f-aa03ab2eeb56 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:24.110969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0dc2b785-c6c7-4acd-806f-aa03ab2eeb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:24.212819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:24.368949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e85855-3020-4149-bec0-801890386d5f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:24.371118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a5e85855-3020-4149-bec0-801890386d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:24.491073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e85855-3020-4149-bec0-801890386d5f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:24.497518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:24.498110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:24.498385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:24.498651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:24.506902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:24.508542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f4bd-9d8e-4821-9b7c-0ac5d9d2e216 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:24.517883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 755/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:25.365696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df263b-593a-4003-86c9-c3287ae1d4d6 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:25.366275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df263b-593a-4003-86c9-c3287ae1d4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:25.366275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df263b-593a-4003-86c9-c3287ae1d4d6 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:25.367609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:25.373304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42fd3-b8ad-4326-b86e-3245f7df2b3a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:25.373723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42fd3-b8ad-4326-b86e-3245f7df2b3a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:25.373842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42fd3-b8ad-4326-b86e-3245f7df2b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:25.392776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-39e42fd3-b8ad-4326-b86e-3245f7df2b3a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:25.393964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-39e42fd3-b8ad-4326-b86e-3245f7df2b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:25.401851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:25.555811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df263b-593a-4003-86c9-c3287ae1d4d6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:25.571971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df263b-593a-4003-86c9-c3287ae1d4d6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:25.577683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:25.578271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:25.578546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:25.578848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:25.586971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:25.593955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0c206-7136-4119-afe2-6e721d213d3d service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:25.598270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 760/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:26.022745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389cf2-ddb6-4159-bfb0-926119f0a0f1 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:26.023026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389cf2-ddb6-4159-bfb0-926119f0a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:26.023298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1389cf2-ddb6-4159-bfb0-926119f0a0f1 None None] PATH_INFO: `/v3` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:26.024597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 757/1517] 10.222.0.84 () {54 vars in 925 bytes} [Tue Aug 9 18:13:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:13:26.031045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb486-5697-43c9-96cc-c6f7def6a97b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:26.031329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb486-5697-43c9-96cc-c6f7def6a97b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:26.031601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb486-5697-43c9-96cc-c6f7def6a97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:26.089535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-400cb486-5697-43c9-96cc-c6f7def6a97b None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:26.090742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-400cb486-5697-43c9-96cc-c6f7def6a97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:26.107511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:26.161209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 761/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Tue Aug 9 18:13:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:30.255514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2a5ff-ec76-4c5b-ac82-e4a3f29eff0c None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:30.256096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2a5ff-ec76-4c5b-ac82-e4a3f29eff0c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:30.256096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2a5ff-ec76-4c5b-ac82-e4a3f29eff0c None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:30.257392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 758/1519] 10.222.0.36 () {54 vars in 961 bytes} [Tue Aug 9 18:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:30.262747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513848f-d92a-41f7-b4b7-eb32a15d286d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:30.263210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513848f-d92a-41f7-b4b7-eb32a15d286d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:30.263313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513848f-d92a-41f7-b4b7-eb32a15d286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:30.284502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e513848f-d92a-41f7-b4b7-eb32a15d286d None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:30.285700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e513848f-d92a-41f7-b4b7-eb32a15d286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:30.294827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 762/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Aug 9 18:13:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:30.302252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2a5ff-ec76-4c5b-ac82-e4a3f29eff0c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:30.315369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2a5ff-ec76-4c5b-ac82-e4a3f29eff0c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:30.320256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:30.320839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:30.321115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:30.321386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:30.329544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:30.331223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df501e5-eec3-4c68-88aa-51b4aa5df6bc service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:30.341288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 759/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:36.804799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b558e8-7c58-4a47-8b99-8ab1e023dac7 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:36.805459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b558e8-7c58-4a47-8b99-8ab1e023dac7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:36.805459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b558e8-7c58-4a47-8b99-8ab1e023dac7 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:36.807285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:36.812184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a91fb-f812-4be0-a8d6-253e8c288565 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:36.812608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a91fb-f812-4be0-a8d6-253e8c288565 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:36.812746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a91fb-f812-4be0-a8d6-253e8c288565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:36.832485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-809a91fb-f812-4be0-a8d6-253e8c288565 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:36.834664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-809a91fb-f812-4be0-a8d6-253e8c288565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:36.842321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:36.998233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b558e8-7c58-4a47-8b99-8ab1e023dac7 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:37.013163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b558e8-7c58-4a47-8b99-8ab1e023dac7 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:37.018745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:37.019360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:37.019635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:37.019907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:37.028178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:37.035176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8466-9ca8-42e6-a510-00f8b0f4fe60 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:37.038990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 764/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:37.874380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f23b-d6e8-4a24-a738-7738639285f0 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:37.874955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f23b-d6e8-4a24-a738-7738639285f0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:37.874955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f23b-d6e8-4a24-a738-7738639285f0 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:37.876309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:37.881040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979f221-e3ed-419c-b134-4ecfe5d9f398 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:37.881445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979f221-e3ed-419c-b134-4ecfe5d9f398 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:37.881605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979f221-e3ed-419c-b134-4ecfe5d9f398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:37.901579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c979f221-e3ed-419c-b134-4ecfe5d9f398 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:37.902791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c979f221-e3ed-419c-b134-4ecfe5d9f398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:37.910456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:38.065367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341f23b-d6e8-4a24-a738-7738639285f0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:38.080532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341f23b-d6e8-4a24-a738-7738639285f0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:38.085957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:38.086543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:38.086872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:38.087120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:38.095910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:38.097486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d0768-13e6-417d-b42a-1cba88610ee5 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:38.107058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 762/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:42.544897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:42.545451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:42.545451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:42.546845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:42.551919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd2ad-21a6-4030-87b7-252cdacf3a13 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:42.552357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd2ad-21a6-4030-87b7-252cdacf3a13 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:42.552574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd2ad-21a6-4030-87b7-252cdacf3a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:42.572871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5effd2ad-21a6-4030-87b7-252cdacf3a13 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:42.574409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5effd2ad-21a6-4030-87b7-252cdacf3a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:42.582224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:42.656992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:42.658724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:42.737100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f5a9e-1b80-491b-a9b1-34e827d7f924 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:42.741132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:42.741729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:42.742009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:42.742277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:42.751998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:42.753460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7f99-d7fb-4d7f-be37-f27a877ad5dd service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:42.762992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 767/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:44.769117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088e2b-873b-4ff1-9941-a79379e3db28 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:44.769656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088e2b-873b-4ff1-9941-a79379e3db28 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:44.769656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088e2b-873b-4ff1-9941-a79379e3db28 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:44.771048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:44.777008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31797b-c54d-4c87-a361-7c4abd076615 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:44.779819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31797b-c54d-4c87-a361-7c4abd076615 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:44.779819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31797b-c54d-4c87-a361-7c4abd076615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:44.797425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ac31797b-c54d-4c87-a361-7c4abd076615 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:44.798599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ac31797b-c54d-4c87-a361-7c4abd076615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:44.806748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:44.843339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088e2b-873b-4ff1-9941-a79379e3db28 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:44.854174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088e2b-873b-4ff1-9941-a79379e3db28 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:44.857680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:44.858292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:44.858564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:44.858874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:44.867576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:44.869254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa420f0f-cb0d-4df6-a025-4df3e4033f38 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:44.878606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 765/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:48.691331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:48.691898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:48.691898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:48.693223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:48.698697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1de06-b2d6-44b1-8781-5baf0c834889 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:48.699146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1de06-b2d6-44b1-8781-5baf0c834889 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:48.699272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1de06-b2d6-44b1-8781-5baf0c834889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:48.718378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e0c1de06-b2d6-44b1-8781-5baf0c834889 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:48.719533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e0c1de06-b2d6-44b1-8781-5baf0c834889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:48.727324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:13:49.032402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:49.034054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:49.098001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:49.110988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd371c-136c-40dc-8b62-18790e3edd2e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:49.114563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:49.115192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:49.115470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:49.115741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:49.125376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:49.126824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0e9fc-2213-4915-abf3-cc5749cbccb7 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:49.137165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 770/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:51.168750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:51.169288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:51.169288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:51.170554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:13:51.175643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab90d-c6bd-4187-873b-588fa35d2700 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:51.176026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab90d-c6bd-4187-873b-588fa35d2700 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:51.176188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab90d-c6bd-4187-873b-588fa35d2700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:51.196035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9e7ab90d-c6bd-4187-873b-588fa35d2700 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:13:51.197270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9e7ab90d-c6bd-4187-873b-588fa35d2700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:51.204969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:13:51.512266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:51.514257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:51.527408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde82949-f7f6-4459-b213-5b1c3fa5356d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:51.530973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:51.531584 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:51.531862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:51.532133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:51.541594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:51.543094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125c7c-ce19-4148-9ea0-e8f3e43a04f1 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:51.552712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 768/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:14:01.191340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=11 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:14:01.548386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:17.428097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.434744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.434744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.457418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.458785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.476953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 772/1540] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:17.485790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.486424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.486424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.528000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.529352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.544618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 769/1541] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:17.549068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:17.564357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43376a7f-161c-4ab8-868b-e6441438a5d9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:17.577891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:17.578792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.579248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.579676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.580490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:17.580995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:17.582775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e5eb-0811-4343-835a-ce56ebc64970 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:17.598032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 773/1542] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:17.606988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:17.620752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b53aa8-d4d1-4be6-b049-ab382e8fe911 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:17.628642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:17.629266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.629706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.629981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.630931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:17.633220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532c49c-d5e9-48ab-9b2a-ff51d8c289b7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:17.655638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b27a-d149-4b4f-b4a4-9cad33c68c4d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.655988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b27a-d149-4b4f-b4a4-9cad33c68c4d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.656195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b27a-d149-4b4f-b4a4-9cad33c68c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.685207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-44c0b27a-d149-4b4f-b4a4-9cad33c68c4d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.686331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-44c0b27a-d149-4b4f-b4a4-9cad33c68c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.704892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 774/1543] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:17.714870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.715195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.715432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.748770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.750530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.760746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 775/1544] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:17.765613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:17.775964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d63c9b-1642-4402-ac9a-b093a973247d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:17.786062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:17.786741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.787228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.787507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.788164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:17.788466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:17.790233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5f24d-71b6-4c47-809d-adfd03f5ec77 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:17.791658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 770/1545] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:17.793244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.793532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.793805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.807535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 776/1546] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:15:17.808803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.809088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.809436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.835775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.837308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.837596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:17.838857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:17.911284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 771/1547] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:17.912391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 777/1548] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:17.912644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:17.913692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:17.929267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed0211-fe4d-4017-95c2-7c97d1ea383b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:17.929502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb82652-d312-4feb-8bb1-d50794508f66 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:17.933881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:17.934158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:17.934479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.934779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:17.934901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.935051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.935156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:17.935374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:17.935673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:17.935933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:17.936684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:17.936684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:15:17.936684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:15:17.936975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:17.936975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:15:17.936975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:15:17.937866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f719-9bd4-4e22-95e6-fa88063a1c89 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:17.938253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35102ac-95d3-4da2-964f-1004a59f71bd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.022340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 772/1549] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.023765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236cc-2188-4773-b40c-7e261a2bb10b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.024055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236cc-2188-4773-b40c-7e261a2bb10b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.024330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236cc-2188-4773-b40c-7e261a2bb10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.043670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-783236cc-2188-4773-b40c-7e261a2bb10b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.044904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-783236cc-2188-4773-b40c-7e261a2bb10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.053305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 778/1550] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:17 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.054654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 773/1551] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.059329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3814a17-89b0-4df5-89f5-de51f6b80c03 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.059329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3814a17-89b0-4df5-89f5-de51f6b80c03 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.059329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3814a17-89b0-4df5-89f5-de51f6b80c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.059640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6ce9-e50a-4022-beb3-7d63799fd060 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.059640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6ce9-e50a-4022-beb3-7d63799fd060 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.059640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6ce9-e50a-4022-beb3-7d63799fd060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.077237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-45ad6ce9-e50a-4022-beb3-7d63799fd060 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.078411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c3814a17-89b0-4df5-89f5-de51f6b80c03 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.078529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-45ad6ce9-e50a-4022-beb3-7d63799fd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.079668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c3814a17-89b0-4df5-89f5-de51f6b80c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.139209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 774/1552] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.140203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 779/1553] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.140501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e781e94-0d17-4aee-9b45-9b0b40b9139c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.140784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e781e94-0d17-4aee-9b45-9b0b40b9139c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.141055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e781e94-0d17-4aee-9b45-9b0b40b9139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.144277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.144848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.145491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.166211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4e781e94-0d17-4aee-9b45-9b0b40b9139c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.166683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.167432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4e781e94-0d17-4aee-9b45-9b0b40b9139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.167881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.175328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 775/1554] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.179831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 780/1555] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.180001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b71d6-f19f-46f3-869d-5681462e0bf4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.180001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b71d6-f19f-46f3-869d-5681462e0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.180199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.180297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b71d6-f19f-46f3-869d-5681462e0bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.189046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b5190-9b2e-4523-8f07-ae703a8adfec None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.193824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.194424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.194750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.195027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.195669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.197265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.198841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-237b71d6-f19f-46f3-869d-5681462e0bf4 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.200029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-237b71d6-f19f-46f3-869d-5681462e0bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.207168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 781/1556] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:18.208023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.212074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 776/1557] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.213332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ce838-2e0f-435b-8369-3c74077a0ad8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.213684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ce838-2e0f-435b-8369-3c74077a0ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.214094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ce838-2e0f-435b-8369-3c74077a0ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.218726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02711f-6bfe-40a7-bf47-5a99848c188d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.223392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.223978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.224809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.225084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.225795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.227765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.233347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-406ce838-2e0f-435b-8369-3c74077a0ad8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.234531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-406ce838-2e0f-435b-8369-3c74077a0ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.243036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 777/1558] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.244235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3ffa-85b5-4eb4-89fe-af2b2305d7d3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.244515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3ffa-85b5-4eb4-89fe-af2b2305d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.244781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3ffa-85b5-4eb4-89fe-af2b2305d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.263628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ef4c3ffa-85b5-4eb4-89fe-af2b2305d7d3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.264801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ef4c3ffa-85b5-4eb4-89fe-af2b2305d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.273360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 778/1559] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.274490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.274813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.275068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.294171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.295453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.303240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 779/1560] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.303996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.312937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3d559-a1c7-4a97-bd22-18bfe45495d8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.317498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.318241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.318653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.318954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.319582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.319878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.320876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:18.320876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:15:18.320876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:15:18.321756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561312d-11ae-4fcb-82cd-b91b7005729e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.323213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:18.331880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:18.335871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 780/1561] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.337504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.337781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.338077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.339670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 782/1562] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.339670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.349198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a528b87-c901-4df8-99cd-009abe9888c8 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.354412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.355028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.355304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.355567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.356190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.357778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.365500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:18.366759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:18.369022 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 783/1563] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:18.370133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.377425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 781/1564] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:18.381955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.382076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18651b81-1452-40e3-877e-875f589cb501 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.387385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.388151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.388430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.388700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.388807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5b83f-db7f-42f7-8671-3f4644bb5dff None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.389343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.389642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.391254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.393431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.394017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.394288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.394553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.395226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.395518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.397617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.412854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 784/1565] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.412854 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.416310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19054ce-c906-4520-af3a-82fc8757c023 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.417419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 782/1566] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.418974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.421226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.421815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.422091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.422364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.423067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.428138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.428138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.432053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb010c-2f89-4f3d-85b1-139a30c843a9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.436851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.437436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.437716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.437982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.438842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.439196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.440797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.442930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 785/1567] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.444529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.457367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fde317-a96a-4580-bbd8-a1f07fe65410 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.457560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 783/1568] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.457560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.461764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.462515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.462783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.463053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.463700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.465349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.472395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36463ecd-4edd-423d-a0fb-82bcacb4059d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.479107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 786/1569] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:18.479107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.480030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.480628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.480903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.481170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.481786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.483749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.488456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48fba5-e962-4463-b8d4-2ca66fcca662 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.494546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.495208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.495485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.495752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] PATH_INFO: `/v3/projects/90a54a6e937e49b9b1da67109827754e/users/e5551cbf3bc04a2db80a6ba72d59cad0/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.511109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a54a6e937e49b9b1da67109827754e, user_id=e5551cbf3bc04a2db80a6ba72d59cad0, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.512142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:18.512142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:15:18.512142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:15:18.514297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.566810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 784/1570] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.567639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.600370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 787/1571] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:18 2022] PUT /identity/v3/projects/90a54a6e937e49b9b1da67109827754e/users/e5551cbf3bc04a2db80a6ba72d59cad0/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:18.605757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.630772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275979d-fc32-4bd5-872e-446d2c3e0e4b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.672018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110332e-18a0-4721-963d-5e735f617660 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.676566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.677202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.678050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.678347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.679045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.680951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.687564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.688224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.688498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.688762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.689472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:18.689697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.692187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.703566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 785/1572] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:18.704347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.713613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049a933-b3d7-4fd0-8d81-0d8cc6e71594 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.718524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.719162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.719440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.719862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.720342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.722264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.766458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 788/1573] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.767387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.818038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 786/1574] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.819044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:18.829877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca100ab-22e5-424e-9631-3cbda8668729 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.839117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:18.843221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:18.884454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223031-d236-4418-bf5e-9ebdbdefc4cb admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:18.888187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.888790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.889035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:18.889295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.889401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.889666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:18.889946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.890063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:18.890242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:18.890925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:18.891898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.892824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:18.996665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 789/1575] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/projects => generated 405 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:18.999666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.027489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 787/1576] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:18 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.027489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.059454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedea523-fb83-4258-ae2f-3d5c6a7184af admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.087180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b69e39-289f-4b09-8d78-a24a601a3de8 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.091886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.092487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.092768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.093039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.093658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.095648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.116884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.119971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.119971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.119971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.119971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.120984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7431a-895c-4bb9-8678-d15da44f499d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.129792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 790/1577] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.131613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.132056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.132334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.159188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] MFA Rules not processed for user `e5551cbf3bc04a2db80a6ba72d59cad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:19.160437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:19.183292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 788/1578] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.184355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.194926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f820bb-e0ea-4b3f-9ee7-80edfc2769d5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.197719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:19.199755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.200345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.200652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.200919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.201530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.203471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.228168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 791/1579] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:19.228989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.239388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb088-81e2-438f-b6a5-5abf859c9d8d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.244841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.245423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.245698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.245962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.246568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.248562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.275639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 789/1580] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.276637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.337014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177b76a-c1ea-49dc-b17b-83136a2312c9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.337379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 792/1581] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.338218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.344134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.344746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.345135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.345451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.346066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.347855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc9ec5-4891-4455-aa3c-df527580bb56 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.348065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.353369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.353954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.354229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.354497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.355137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.357062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.406354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:19.411774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 790/1582] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.423186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.423186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67892774-1a3f-475b-8567-2afc0f9926aa admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.427253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.428482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.428978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.429385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.429994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.431975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.439840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 793/1583] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.441844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.450223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b7fa-3e96-4c14-9f35-4e47291e63aa admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.457431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.457431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.457431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.457431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] PATH_INFO: `/v3/projects/e9b71c6efb004375985eabcb2dd3678e/users/7ec9c7e425784cc2af0b9c4f47463a87/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.471682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b71c6efb004375985eabcb2dd3678e, user_id=7ec9c7e425784cc2af0b9c4f47463a87, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.474385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.513398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 791/1584] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.514148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.523650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb73c-d25d-4686-b0b1-9594d21fc933 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.528190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.528864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.529098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.529594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.530408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.532342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.543594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 792/1585] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.560717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 794/1586] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:19 2022] PUT /identity/v3/projects/e9b71c6efb004375985eabcb2dd3678e/users/7ec9c7e425784cc2af0b9c4f47463a87/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:19.560891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.560993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.561093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fc257-3cdd-49c4-97be-c66209454131 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.610587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377eeb-8702-4050-acac-288f131d7f4c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.616233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.616836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.617112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.617263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.617748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.617856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.617979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.618116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.618384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.619030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.619932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.620948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.714618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 795/1587] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.716389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.726150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff4fe-a9c4-4e17-93aa-03113d769cf6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.731128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.731702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.732020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.732289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.733259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.734879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.742892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 793/1588] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:19.742892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.746722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 796/1589] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.747668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.758048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7a07-a468-4457-a258-78764c23389a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.803880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53aea4-a1db-4240-8fc6-20207edb7cac admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.807783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:19.808526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.809495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.809784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.810040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.810674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.812303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.814453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.815069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.815352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.815614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.816235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.817881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.822560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 794/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.823248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.827954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 797/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.828755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.833079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87cc0-98d8-4a30-ae53-b9baf9d022d6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.838625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.839077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69311e-e35d-49ff-acc2-c3944566656c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.839258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.839531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.839802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.840500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.842001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571c252-255b-44a1-8af2-993f34cb851f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.845047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.845516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.845824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.846211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.846909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.849002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.852834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 795/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.854031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.854314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.854583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.863934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 798/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.863934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.872338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b1073-95e3-42bc-ad73-fc7788ff49a6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.876885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.877447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.877721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.877985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.878605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.880239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.882577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] MFA Rules not processed for user `7ec9c7e425784cc2af0b9c4f47463a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:19.883810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:19.890276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 799/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.891075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.901475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6e5e3-fced-4f5f-af8b-c1aeac53a8b3 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.906131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.906734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.907010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.907282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.907889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.909905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.917212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:19.951827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 796/1595] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:19.952651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.963310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e0cc2-7f54-4e48-943c-acc9a3bc3aa2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.967753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.968329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:19.968601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:19.968862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:19.969488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:19.971100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:19.981842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 797/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:19.982635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:19.994820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2e6a-90fa-4d60-af60-312f10967769 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:19.999333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:19.999941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.000214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.000476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.001095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.002665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.004833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 800/1597] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:19 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:20.005658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.015024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 798/1598] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:20.015967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc8b8-9174-41d2-95f3-28f68c10db7b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.018808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.020866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.022146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.022429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.022696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.023361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.024943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.029429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a5a8f-5104-4713-b576-352217907c5a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.034056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.034630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.034931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.035200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.035745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 801/1599] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:20.035879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.036519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.037614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.046206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc255c-441a-4e4f-965a-8cc86b245050 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.048434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 799/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:20.049152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.051133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.051964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.052104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.052297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] PATH_INFO: `/v3/projects/ac2e5c4bafff4b16bb9d82dacb5dd056/users/528811040ede40bdb40e3e6e666d283d/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.059001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7210a-8be9-4e5d-aa44-77c7ef862a9b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.063821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.064395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.064666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.064931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.065557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.067165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f248e-bc17-4d88-aa82-1658c8658603 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.068221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2e5c4bafff4b16bb9d82dacb5dd056, user_id=528811040ede40bdb40e3e6e666d283d, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.071421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 800/1601] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e830-5e3f-473c-94be-d2a4b261864a None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e830-5e3f-473c-94be-d2a4b261864a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e830-5e3f-473c-94be-d2a4b261864a None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 801/1602] 10.222.0.36 () {54 vars in 962 bytes} [Tue Aug 9 18:15:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:20.430062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207e830-5e3f-473c-94be-d2a4b261864a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.430621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207e830-5e3f-473c-94be-d2a4b261864a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.430621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.430621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.430621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.431009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] PATH_INFO: `/v3/projects/aa4976a9e83745b1851277848cc99c37/users/7569e6a68c9d40709954daf6a5e40c08/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.431009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4976a9e83745b1851277848cc99c37, user_id=7569e6a68c9d40709954daf6a5e40c08, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.431009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.497859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 802/1603] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/ac2e5c4bafff4b16bb9d82dacb5dd056/users/528811040ede40bdb40e3e6e666d283d/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:20.498811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.519305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 802/1604] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/aa4976a9e83745b1851277848cc99c37/users/7569e6a68c9d40709954daf6a5e40c08/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:20.520261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.571320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7c6f2-4591-4ebc-9508-ff00be5889cb admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.583748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c7d32-6380-4586-a53a-e8810d21b424 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.589420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.594348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.594348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.594348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] PATH_INFO: `/v3/projects/ad80b76c328d44aa9479a98367f0f65c/users/5f405a8b7cd64c7e9615a528a0d2484b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.605824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad80b76c328d44aa9479a98367f0f65c, user_id=5f405a8b7cd64c7e9615a528a0d2484b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.608532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.630007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.630634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.631403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.631538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.632168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.633763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.643349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 803/1605] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:20.647445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.653963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe8-6d4a-49c8-8c70-067302643dcf admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.658553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.659318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.659596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.659869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] PATH_INFO: `/v3/projects/b6b439503fe44230b9ede8923885a231/users/e154686f161c41d3a03b1e589d95e1f5/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.673277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 803/1606] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/ad80b76c328d44aa9479a98367f0f65c/users/5f405a8b7cd64c7e9615a528a0d2484b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:20.674803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.678949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b439503fe44230b9ede8923885a231, user_id=e154686f161c41d3a03b1e589d95e1f5, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.681694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fde2e1-05ca-45cf-baff-48bb40ae2dc3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.685043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:20.723465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:20.738377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaff1d6-aad4-4993-9b92-85dd2d06ac29 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.743655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.745012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.745012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.745012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] PATH_INFO: `/v3/projects/8e07f9d1cb104e298395fd112326f00f/users/6ff88a42c4f24675b77406235e61c294/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.758942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 804/1607] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/b6b439503fe44230b9ede8923885a231/users/e154686f161c41d3a03b1e589d95e1f5/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:20.758942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab439c-3d9a-49f9-8d60-12171e0e4170 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.758942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab439c-3d9a-49f9-8d60-12171e0e4170 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.758942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab439c-3d9a-49f9-8d60-12171e0e4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.760252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e07f9d1cb104e298395fd112326f00f, user_id=6ff88a42c4f24675b77406235e61c294, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.763029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.797647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cdab439c-3d9a-49f9-8d60-12171e0e4170 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:20.799054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cdab439c-3d9a-49f9-8d60-12171e0e4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:20.851513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 804/1608] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/8e07f9d1cb104e298395fd112326f00f/users/6ff88a42c4f24675b77406235e61c294/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:20.852637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:20.870984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 805/1609] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Aug 9 18:15:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:20.872341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3b01-a565-453a-b155-12c584230b7e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.872552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3b01-a565-453a-b155-12c584230b7e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.872822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3b01-a565-453a-b155-12c584230b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.892566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-582f3b01-a565-453a-b155-12c584230b7e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:20.893742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-582f3b01-a565-453a-b155-12c584230b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:20.929442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea544bf-2011-4607-8768-93bc6662b48d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:20.934647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:20.935283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.935557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.935823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] PATH_INFO: `/v3/projects/dbd7e50444cc4ef0873bfac9d9a2d514/users/18fae6990e4148ff97dec28b204c52df/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.951063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd7e50444cc4ef0873bfac9d9a2d514, user_id=18fae6990e4148ff97dec28b204c52df, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:20.953463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 806/1610] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:20.954771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9418-64d7-4126-9d4b-0f736832501e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:20.955058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9418-64d7-4126-9d4b-0f736832501e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:20.955328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9418-64d7-4126-9d4b-0f736832501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:20.956920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:20.983846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0a3f9418-64d7-4126-9d4b-0f736832501e None None] MFA Rules not processed for user `7569e6a68c9d40709954daf6a5e40c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:20.985041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0a3f9418-64d7-4126-9d4b-0f736832501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.023504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 805/1611] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:20 2022] PUT /identity/v3/projects/dbd7e50444cc4ef0873bfac9d9a2d514/users/18fae6990e4148ff97dec28b204c52df/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:21.024557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.051575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 807/1612] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.054036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9ccd-8f1f-4144-b678-ba7a6241c0ef None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.054420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9ccd-8f1f-4144-b678-ba7a6241c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.054813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9ccd-8f1f-4144-b678-ba7a6241c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.066763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:21.079873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-da0c9ccd-8f1f-4144-b678-ba7a6241c0ef None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:21.081080 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-da0c9ccd-8f1f-4144-b678-ba7a6241c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.093664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf57ad-68d6-4c0c-a5ef-a19e60a96d04 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.098656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.099523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.100012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.100290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] PATH_INFO: `/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6/users/c68489eddb854f24b6764ddb84944977/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.115578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7607d35fe78f4ff6a1d26acdfb036ae6, user_id=c68489eddb854f24b6764ddb84944977, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.119581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924945d-3d4d-4fe4-a85f-8c49d5907327 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.145642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 808/1613] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.147848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.148191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.148522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.176103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] MFA Rules not processed for user `e154686f161c41d3a03b1e589d95e1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:21.177427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.184047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 806/1614] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:21 2022] PUT /identity/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6/users/c68489eddb854f24b6764ddb84944977/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:21.188095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.188095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.188095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.216510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] MFA Rules not processed for user `6ff88a42c4f24675b77406235e61c294`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:21.217725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.244151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:21.246883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 809/1615] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.247765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.290020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 807/1616] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.290913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.292179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.320930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94d43-1f7e-4102-9365-49300c271faa None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.327583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.328263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.328537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.328804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.352344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea758e-2ac0-4318-ae16-e39ed5ee063c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.356174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.359320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.359320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.359320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.361225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:21.390535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.394254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e677874-eb09-40b3-b1bc-ffed72569571 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.402055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 810/1617] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:21.403528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cb57-be9c-467b-9b36-48f88f1d5fbb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.403956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cb57-be9c-467b-9b36-48f88f1d5fbb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.404199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cb57-be9c-467b-9b36-48f88f1d5fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.423091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c922cb57-be9c-467b-9b36-48f88f1d5fbb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:21.428364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.428551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c922cb57-be9c-467b-9b36-48f88f1d5fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.428686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.436990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 811/1618] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.436990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044350d-2c54-4468-bf2a-0af033fa2323 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.436990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044350d-2c54-4468-bf2a-0af033fa2323 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.436990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044350d-2c54-4468-bf2a-0af033fa2323 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.437435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 812/1619] 10.222.0.36 () {54 vars in 962 bytes} [Tue Aug 9 18:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:21.438918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044350d-2c54-4468-bf2a-0af033fa2323 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.446986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 808/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:21.449309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.455506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044350d-2c54-4468-bf2a-0af033fa2323 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.460151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.461301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.462281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.462885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.464328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494aae-9e7a-4ca7-8f90-384e1dcabf8e service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.470624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.471470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.471944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.472392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.473334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.475713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.477609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.479170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.491823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 809/1621] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:21.492723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 813/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:21.493613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.498293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.503716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6653-2d8c-43ce-a46c-cd42ba17865b service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.509435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8af04-e1cf-41df-927c-7607508b1b97 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.514215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.514215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.514215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.514215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.514596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.514596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe853-8bed-4749-8db0-5b4e33c2fab9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.515510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.516292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.516779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.517196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.518024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.520190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.636711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 814/1623] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:21.638477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.638891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.639266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.664024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:21.667894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 810/1624] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:21.668074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:21.668174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.725456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:21.736315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63803-8c01-4720-b888-d5b029928187 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.741008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.741604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.741876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.742141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.750757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 815/1625] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:21.751983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.815105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.818136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.827436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac514d7a-a958-4005-8d71-0e6b7feaa800 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.829919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 811/1626] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:21.830754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.833110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.834017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.835137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.836098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] PATH_INFO: `/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6/users/c68489eddb854f24b6764ddb84944977/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.838025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7607d35fe78f4ff6a1d26acdfb036ae6, user_id=c68489eddb854f24b6764ddb84944977, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.840977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711da6c0-0c82-41f0-aaeb-de74e70161e7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.841812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefb664-c15a-4524-8b47-d66e39de0041 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.848766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.849669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.851052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.851052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.852234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.854296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.854947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 816/1627] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:21 2022] PUT /identity/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6/users/c68489eddb854f24b6764ddb84944977/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:21.856311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca846c4-46c3-4491-b02c-bf9fdef229ab None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.856596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca846c4-46c3-4491-b02c-bf9fdef229ab None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.856869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca846c4-46c3-4491-b02c-bf9fdef229ab None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.858605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 817/1628] 10.222.0.36 () {54 vars in 912 bytes} [Tue Aug 9 18:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:21.859168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca846c4-46c3-4491-b02c-bf9fdef229ab None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.883078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:21.936163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca846c4-46c3-4491-b02c-bf9fdef229ab None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:21.944749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 812/1629] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:21.945934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:21.950004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:21.950630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:21.950989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:21.951267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:21.951913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:21.954264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad381226-d740-47fb-8bd6-b9636dbe004b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:21.963328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f651e8b-e59a-4b23-a4df-598b11644773 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.036655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.037428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.037749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.038020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.042848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 818/1630] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:21 2022] POST /identity/v3/users => generated 308 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:22.044288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.044578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.044853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.076046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] MFA Rules not processed for user `c68489eddb854f24b6764ddb84944977`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:22.077486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:22.102976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.104635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb297b4-1e28-4eb0-be75-83a145b094a7 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.114333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 813/1631] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:22.117652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.117652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.117652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.137310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:22.138556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:22.157264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 814/1632] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:15:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:22.158158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.169710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 819/1633] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:22.172648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.181129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:22.189275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455656e2-b6fe-4bcf-94de-c72e6cd22ab9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.201030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.201150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b551989-b88a-403d-b6d7-38638ce99a71 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.202124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.202796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.203076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.205315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.215602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.215850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.215850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.216011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.217375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.218382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.221942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.237091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 815/1634] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:22.239558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.251547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 820/1635] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:22.252881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.319299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e6770-2eca-491c-94d5-c0ccfd95bf73 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.324400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.325061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.325338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.325762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.338784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.340298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.342015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72cabc-3b4c-437d-9933-d2f62659f7ac admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.351473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.352939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.353679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.353955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.358621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 816/1636] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:22.359537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.373621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11882ef-4224-4fc7-b713-62e0360b19bf service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.376187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.377935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.379627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.380549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.382023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.382372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] PATH_INFO: `/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d/users/5af99d9358ad4529868cdfdc6dbc4ba2/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.403576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a968bfc7e642f4bf83f6c5e87ce04d, user_id=5af99d9358ad4529868cdfdc6dbc4ba2, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.406836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.408429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 821/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:22.409336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.428355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb0b20-9cb0-47b0-bd03-ab2542771585 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.433765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.434292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.434589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.434899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] PATH_INFO: `/v3/projects/5840ab71f44f4c0c9537afac20f67742/users/eeaa02860d8d4e1097c87d62ff601e18/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.459798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5840ab71f44f4c0c9537afac20f67742, user_id=eeaa02860d8d4e1097c87d62ff601e18, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.466423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a83d1-f8bb-4213-a328-2e4f5ea53a05 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.505166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 817/1638] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:22 2022] PUT /identity/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d/users/5af99d9358ad4529868cdfdc6dbc4ba2/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:22.508790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.598091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72e2-faea-4de9-89aa-9de8f5c2fd0e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.606051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.606893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.607501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.607846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.609035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.613055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:22.622742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 822/1639] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:22 2022] PUT /identity/v3/projects/5840ab71f44f4c0c9537afac20f67742/users/eeaa02860d8d4e1097c87d62ff601e18/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:22.622742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa133-aec3-4ecf-a8bf-539afbc7661a None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.622742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa133-aec3-4ecf-a8bf-539afbc7661a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.622742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa133-aec3-4ecf-a8bf-539afbc7661a None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.624181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 823/1640] 10.222.0.36 () {54 vars in 912 bytes} [Tue Aug 9 18:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:22.624303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248732b6-d1cb-4c40-bf3e-0289f7c2b8fe admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.626013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffa133-aec3-4ecf-a8bf-539afbc7661a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.652768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 818/1641] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:22.653790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.654226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.654676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.683216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:22.684448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:22.783867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffa133-aec3-4ecf-a8bf-539afbc7661a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.793574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.793574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.793574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.793574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.794046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.794046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.799819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:22.810808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 824/1642] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:22.810808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.821630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c9db4-f354-41ed-a9d8-3d8674c951b7 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.822361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 819/1643] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:15:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:22.823144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.850109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.852790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.853076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.854123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] PATH_INFO: `/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d/users/5af99d9358ad4529868cdfdc6dbc4ba2/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:22.860612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a968bfc7e642f4bf83f6c5e87ce04d, user_id=5af99d9358ad4529868cdfdc6dbc4ba2, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:22.869436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:22.890798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 825/1644] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:22 2022] PUT /identity/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d/users/5af99d9358ad4529868cdfdc6dbc4ba2/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:22.891768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:22.928784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eded4-70c3-4c3e-8d4d-2693f7f6fc22 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:22.992179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:22.992693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:22.992982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:22.993239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.006129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e8f6-426e-4748-a18b-73c577c42dd6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:23.013706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:23.014395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.014746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.015118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] PATH_INFO: `/v3/projects/5840ab71f44f4c0c9537afac20f67742/users/eeaa02860d8d4e1097c87d62ff601e18/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.017325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5840ab71f44f4c0c9537afac20f67742, user_id=eeaa02860d8d4e1097c87d62ff601e18, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:23.020115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568916be-fcc8-4781-9e47-fd6c2b01f9c4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:23.047498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 826/1645] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:22 2022] PUT /identity/v3/projects/5840ab71f44f4c0c9537afac20f67742/users/eeaa02860d8d4e1097c87d62ff601e18/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:23.047498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.047498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.047498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.058987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:23.060450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfaf9-df45-4215-a834-dc29f9d6c2c6 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:23.081271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 820/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:23.082741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.088994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.088994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.108469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] MFA Rules not processed for user `5af99d9358ad4529868cdfdc6dbc4ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:23.110635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:23.130130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] MFA Rules not processed for user `eeaa02860d8d4e1097c87d62ff601e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:23.132567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:23.217868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:23.222289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 827/1647] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:23.229069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 821/1648] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:23.497482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:23.501603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:23.561223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693638f1-a757-42ce-813d-98dd5fe929d2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:23.565662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:23.566290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.567054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.567386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.666689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:23.669364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:23.687864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 828/1649] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:23.745177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:23.767750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3497a8f-058c-4d5c-8fff-56e20d59d24c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:23.777967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:23.778983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.779476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.779902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.793421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:23.795847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:23.827357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 822/1650] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:23.950112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:23.962134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa7a07-eab2-447e-ac6b-a93542129dc0 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:23.968846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:23.969814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:23.970162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:23.970691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:23.983486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:23.985508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:24.006905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 829/1651] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:23 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:24.612722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:24.625380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817ded2-1a15-4976-9e9d-5da13609967d service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:24.635547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:24.636538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:24.637009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:24.637556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:24.722785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:24.725419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c9eef-0106-44e8-aae9-2a94c74ec9f5 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:24.744095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 823/1652] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:24.977329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:25.124847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017134-1ca8-402b-8d2b-faa5665868b7 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:25.133502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:25.134913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:25.135541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:25.136267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] PATH_INFO: `/v3/users/e154686f161c41d3a03b1e589d95e1f5` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:25.138087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e154686f161c41d3a03b1e589d95e1f5)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:25.140724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: WARNING py.warnings [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:25.140724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: warnings.warn(msg) Aug 09 18:15:25.140724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: Aug 09 18:15:25.143126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:25.420291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:25.476220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e154686f161c41d3a03b1e589d95e1f5'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:25.526353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e154686f161c41d3a03b1e589d95e1f5'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:25.527279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e154686f161c41d3a03b1e589d95e1f5'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:25.546953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e154686f161c41d3a03b1e589d95e1f5'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:25.550162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e154686f161c41d3a03b1e589d95e1f5'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:25.683668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 830/1653] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:24 2022] DELETE /identity/v3/users/e154686f161c41d3a03b1e589d95e1f5 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:25.686284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:25.686787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:25.687191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:25.747145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:25.749685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:25.991685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 824/1654] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:26.078935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:26.080833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:26.220075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17b7e-4994-4eb5-884f-1a2e5a1dee45 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:26.225077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:26.225765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:26.226046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:26.226322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:26.244626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:26.246376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:26.340105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 831/1655] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:27.035074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:27.035074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:27.086102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdec7b2-4644-4add-a6f6-96fc54f05193 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:27.105990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:27.108359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:27.109574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:27.110095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] PATH_INFO: `/v3/users/7ec9c7e425784cc2af0b9c4f47463a87` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:27.136755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec9c7e425784cc2af0b9c4f47463a87)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:27.138744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:27.377936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:27.379124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:27.428406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5eaa0-d766-41eb-a911-6ed4278fe2d5 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:27.447891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:27.477380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:27.484532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:27.494915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] PATH_INFO: `/v3/projects/b6b439503fe44230b9ede8923885a231` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:27.494915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b439503fe44230b9ede8923885a231)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:27.504822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:27.511529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec9c7e425784cc2af0b9c4f47463a87'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.541739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec9c7e425784cc2af0b9c4f47463a87'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.597618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec9c7e425784cc2af0b9c4f47463a87'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.602344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:27.757107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec9c7e425784cc2af0b9c4f47463a87'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.767092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c1e21637-42be-4fb5-937e-fd8b411fe5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec9c7e425784cc2af0b9c4f47463a87'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.779624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 825/1656] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:27 2022] DELETE /identity/v3/users/7ec9c7e425784cc2af0b9c4f47463a87 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:27.788484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:27.789025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:27.789418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:27.829802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b439503fe44230b9ede8923885a231', 'request_id': 'req-d9fafc99-c16b-4cab-b0fb-da20a883cec9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.854696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b439503fe44230b9ede8923885a231', 'request_id': 'req-d9fafc99-c16b-4cab-b0fb-da20a883cec9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:27.871196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:27.876064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:27.887268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b439503fe44230b9ede8923885a231', 'request_id': 'req-d9fafc99-c16b-4cab-b0fb-da20a883cec9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.011729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] The token cache is being invalidate because project b6b439503fe44230b9ede8923885a231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:28.012941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.013648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.019901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 832/1657] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:27 2022] DELETE /identity/v3/projects/b6b439503fe44230b9ede8923885a231 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:28.025576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:28.028324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:28.066333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 826/1658] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:28.105994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:28.150839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fafc99-c16b-4cab-b0fb-da20a883cec9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:28.153706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:28.154735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:28.155108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:28.155446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] PATH_INFO: `/v3/users/eeaa02860d8d4e1097c87d62ff601e18` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:28.156733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeaa02860d8d4e1097c87d62ff601e18)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:28.164208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:28.213019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:28.214781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:28.359010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926d83-e28a-4af9-af70-9b352bd89eb0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:28.374522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:28.375746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:28.376274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:28.377462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:28.423528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:28.424786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeaa02860d8d4e1097c87d62ff601e18'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.426385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:28.458001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 827/1659] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:28.459953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeaa02860d8d4e1097c87d62ff601e18'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.461471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeaa02860d8d4e1097c87d62ff601e18'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.472343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeaa02860d8d4e1097c87d62ff601e18'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.475090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-54c4c530-9ef4-40c4-b5bc-8502f5e878ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeaa02860d8d4e1097c87d62ff601e18'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:28.544299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 833/1660] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:28 2022] DELETE /identity/v3/users/eeaa02860d8d4e1097c87d62ff601e18 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:28.856409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:28.857935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:28.889323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510276b-c385-4225-9b21-7fa27537fa03 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:29.047672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:29.047672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:29.047672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:29.047672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] PATH_INFO: `/v3/users/e5551cbf3bc04a2db80a6ba72d59cad0` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:29.048128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5551cbf3bc04a2db80a6ba72d59cad0)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:29.049280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:29.273744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:29.288356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5551cbf3bc04a2db80a6ba72d59cad0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.309066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5551cbf3bc04a2db80a6ba72d59cad0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.342692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5551cbf3bc04a2db80a6ba72d59cad0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.429699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5551cbf3bc04a2db80a6ba72d59cad0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.430875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5551cbf3bc04a2db80a6ba72d59cad0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.441227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 828/1661] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:28 2022] DELETE /identity/v3/users/e5551cbf3bc04a2db80a6ba72d59cad0 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:29.445939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846663-92e0-4861-b717-14b0585192ea None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:29.446456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846663-92e0-4861-b717-14b0585192ea None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:29.446584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846663-92e0-4861-b717-14b0585192ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:29.492240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7f846663-92e0-4861-b717-14b0585192ea None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:29.493821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7f846663-92e0-4861-b717-14b0585192ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:29.599811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 834/1662] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:29.624299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:29.625802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:29.743683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846663-92e0-4861-b717-14b0585192ea None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:29.745229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7f846663-92e0-4861-b717-14b0585192ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:29.751393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72945ee-5929-4964-a2f7-f304e0e315fa admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:29.760759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:29.761762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:29.761762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:29.761911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:29.773255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846663-92e0-4861-b717-14b0585192ea None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:29.783382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:29.787881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:29.788218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:29.788453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] PATH_INFO: `/v3/projects/5840ab71f44f4c0c9537afac20f67742` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:29.796035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:29.797693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:29.799024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5840ab71f44f4c0c9537afac20f67742)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:29.803881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:29.822880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 829/1663] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:29.955063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:29.957625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5840ab71f44f4c0c9537afac20f67742', 'request_id': 'req-74622b7a-746a-4463-a2b8-d6d61cb4e75f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.966248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5840ab71f44f4c0c9537afac20f67742', 'request_id': 'req-74622b7a-746a-4463-a2b8-d6d61cb4e75f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.974959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5840ab71f44f4c0c9537afac20f67742', 'request_id': 'req-74622b7a-746a-4463-a2b8-d6d61cb4e75f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:29.999640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.000533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.026607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be2e07-832e-49e6-a9dc-baea03454e2d service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:30.078936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] The token cache is being invalidate because project 5840ab71f44f4c0c9537afac20f67742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:30.079485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.080304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-74622b7a-746a-4463-a2b8-d6d61cb4e75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.086677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 835/1664] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:29 2022] DELETE /identity/v3/projects/5840ab71f44f4c0c9537afac20f67742 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:30.092698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87fd8-14e7-41a4-88fb-fb639fa528be None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.093216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87fd8-14e7-41a4-88fb-fb639fa528be None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.093807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87fd8-14e7-41a4-88fb-fb639fa528be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.150564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3cb87fd8-14e7-41a4-88fb-fb639fa528be None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:30.154633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3cb87fd8-14e7-41a4-88fb-fb639fa528be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.168888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:30.169580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.170030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.170454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] PATH_INFO: `/v3/projects/e9b71c6efb004375985eabcb2dd3678e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.172682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b71c6efb004375985eabcb2dd3678e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:30.173731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: WARNING py.warnings [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:15:30.173731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: warnings.warn(msg) Aug 09 18:15:30.173731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: Aug 09 18:15:30.174967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:30.320608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 836/1665] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:30.328729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.338741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.339355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.387567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b71c6efb004375985eabcb2dd3678e', 'request_id': 'req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.388504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:30.390289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.406065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b71c6efb004375985eabcb2dd3678e', 'request_id': 'req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.421075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:30.434021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b71c6efb004375985eabcb2dd3678e', 'request_id': 'req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.556588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 837/1666] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:30.560976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.570733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] The token cache is being invalidate because project e9b71c6efb004375985eabcb2dd3678e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:30.571163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.575471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-84d3b36c-92e2-425b-8ce0-eabe9860f2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:30.579226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 830/1667] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:29 2022] DELETE /identity/v3/projects/e9b71c6efb004375985eabcb2dd3678e => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:30.579780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4de0f-d6dd-4b1c-bc31-bbb26ee4c6eb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:30.582306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fbeb2-0c86-450b-9589-29e3a1330491 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.582521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fbeb2-0c86-450b-9589-29e3a1330491 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.582822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fbeb2-0c86-450b-9589-29e3a1330491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.587943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:30.588900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.589230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.589503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.590313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:30.590740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:30.592963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:30.615929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 838/1668] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:30.621086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.625200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.638556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7b8fbeb2-0c86-450b-9589-29e3a1330491 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:30.639564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7b8fbeb2-0c86-450b-9589-29e3a1330491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.659602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 831/1669] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:30.665411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7485c03-1168-418e-ab9c-9aacc06f53aa admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:30.667646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.667784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.668045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.680426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:30.683273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.683273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.683273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.685392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:30.690811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:30.702589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:30.706102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:30.733149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 832/1670] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:30.737933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.779919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331f1a-adc6-4927-9331-d76023db32f4 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:30.790523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:30.791008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.791288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.791564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.792724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:30.793026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:30.796989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:30.815116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 833/1671] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:30.819654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.858581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54b40a-d47a-410d-81df-30b7e1ff5018 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:30.868011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:30.870478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:30.870949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:30.871378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:30.872482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:30.875779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:30.880157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 839/1672] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/projects => generated 357 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:30.883516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:30.992470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:31.008056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 834/1673] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:31.013861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.033777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bad82-1220-4069-85cb-565fc9f8ba94 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.155351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.155962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.156952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.156952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.158485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.160499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.181026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0bd45-0bed-4ce3-88cd-747832309a3f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.195888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.196519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.197037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.197258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.200046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.202214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.310810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 840/1674] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:30 2022] POST /identity/v3/users => generated 279 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:15:31.316784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.317405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:31.343698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1a87b-e874-468a-b405-7b2e00a79954 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.345600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 835/1675] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:31 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:31.346790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.366895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.368499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.368499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.368499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] PATH_INFO: `/v3/projects/90a54a6e937e49b9b1da67109827754e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.369536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a54a6e937e49b9b1da67109827754e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.371858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.372317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be92bd-c2cf-4a65-a990-400076eaba18 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.382867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.383646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.384078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.384631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.385430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.392238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.439486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 836/1676] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:31.439486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.477915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c1705-58ce-43b9-9956-07fbaba2aa69 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.508191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.508993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.509382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.509938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.510305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.512069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.544336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 837/1677] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:31.545334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.559095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a54a6e937e49b9b1da67109827754e', 'request_id': 'req-0b4b6208-7e2d-44ee-9027-ae574da07673'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:31.573679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24286779-ab1e-4874-83c8-53b994843c85 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.584116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a54a6e937e49b9b1da67109827754e', 'request_id': 'req-0b4b6208-7e2d-44ee-9027-ae574da07673'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:31.602752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a54a6e937e49b9b1da67109827754e', 'request_id': 'req-0b4b6208-7e2d-44ee-9027-ae574da07673'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:31.720678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] The token cache is being invalidate because project 90a54a6e937e49b9b1da67109827754e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:31.721185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:31.722119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.722649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:31.723621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.723621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.723621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.724917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.728418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 841/1678] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:31 2022] DELETE /identity/v3/projects/90a54a6e937e49b9b1da67109827754e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:31.729758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.732305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:31.738902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.767372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 838/1679] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:31.768026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6208-7e2d-44ee-9027-ae574da07673 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.768142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.772446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:31.783318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.785217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:31.787385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.787829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.790414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.792458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96163d9-1a98-4432-92de-5c5cf9cb1b2d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.794245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.798075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.798831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.799476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.799752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.800029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] PATH_INFO: `/v3/users/c68489eddb854f24b6764ddb84944977` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.801581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68489eddb854f24b6764ddb84944977)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.803808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:31.830624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 842/1680] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:31.836811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:31.839256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:31.860431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7dab-3072-4891-9127-0ee5bea97e5c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:31.867179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:31.867179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:31.867179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:31.867938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] PATH_INFO: `/v3/projects/e2a41d2c4ff343e59f3ca24bbdae5e41/users/cbd99565107047e5a2a392ae7297a1a4/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:31.885539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a41d2c4ff343e59f3ca24bbdae5e41, user_id=cbd99565107047e5a2a392ae7297a1a4, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:31.888866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:32.021068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 843/1681] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:31 2022] PUT /identity/v3/projects/e2a41d2c4ff343e59f3ca24bbdae5e41/users/cbd99565107047e5a2a392ae7297a1a4/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:32.021068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:32.057579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68489eddb854f24b6764ddb84944977'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:32.066449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68489eddb854f24b6764ddb84944977'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:32.081826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:32.083426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68489eddb854f24b6764ddb84944977'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:32.153376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68489eddb854f24b6764ddb84944977'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:32.154278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5736a5a8-5b92-4d6e-a834-b88044161778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68489eddb854f24b6764ddb84944977'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:32.160317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f3db2-d9cd-4f06-a7ad-db632aa50b0d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:32.164101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 839/1682] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:31 2022] DELETE /identity/v3/users/c68489eddb854f24b6764ddb84944977 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:32.167781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:32.168327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:32.169438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:32.258251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] MFA Rules not processed for user `cbd99565107047e5a2a392ae7297a1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:32.262046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:32.305203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:32.306226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:32.306857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:32.307878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] PATH_INFO: `/v3/projects/a636a52498504f69a9484cbe8d7a1c3a/users/0d9edcdcbae74071bbf54a0904d00d2c/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:32.336064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a636a52498504f69a9484cbe8d7a1c3a, user_id=0d9edcdcbae74071bbf54a0904d00d2c, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:32.340990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00895e-77e5-4c05-853d-fb17e6068cfc admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:32.446304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 844/1683] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:32 2022] PUT /identity/v3/projects/a636a52498504f69a9484cbe8d7a1c3a/users/0d9edcdcbae74071bbf54a0904d00d2c/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:32.459446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:32.459723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:32.459979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:32.462399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 840/1684] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:32.466885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:32.470970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:32.508429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] MFA Rules not processed for user `0d9edcdcbae74071bbf54a0904d00d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:32.509599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea7fe-2c1b-4efe-9d12-505d9149bbb2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:32.510853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:32.642154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:32.642806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:32.643088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:32.643361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:32.643994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:32.646678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:32.671640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:32.690985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 845/1685] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:32.705168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:32.710050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:32.743944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5c5e-017d-424d-9aa5-9fa2644835f1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:32.756633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:32.758976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:32.759437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:32.759872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:32.760661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:32.763076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:32.799700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 841/1686] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/projects => generated 357 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:15:32.800604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:32.882159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 846/1687] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:32.886102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:32.891270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a8e-f67e-46e9-a91d-223840bccd6c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:32.952753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:33.058142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7803ad-f1c8-4942-941c-e6b504dd9199 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.065647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.066260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.066532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.066831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.070973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.074026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.074909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.075128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.075402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.076908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.077305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.084715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.198896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 842/1688] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/users => generated 285 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:15:33.203426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.218874 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c30df-91f4-4784-a74d-bca34dcea628 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.244872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 847/1689] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:32 2022] POST /identity/v3/users => generated 289 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:33.249901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.280064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.288180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.288912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.294048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.297065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.305540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.320628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54f83b-3270-4e7a-80b9-acb9939c5a13 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.350804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.350804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.350804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.350804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.351228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.351353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 843/1690] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:33.351601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.351811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.393283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a5115-ade7-40f6-9de9-d191f2d274fc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.395208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 848/1691] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:33.407373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.408519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.409377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.409802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.410445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] PATH_INFO: `/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83/users/7268b8dc2f94404eb132f8cd8f8cec0d/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.423445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e3421-a7b1-4234-96bb-42ccf2d68d74 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.436789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.437072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.437339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.437629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] PATH_INFO: `/v3/projects/400010b050a44efab2072afb4b22edd9/users/5883efbb4e634c67b33fab9ec9f3bacc/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.466055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b881de0f39a4dbf9a8fdfa85bea8b83, user_id=7268b8dc2f94404eb132f8cd8f8cec0d, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.477095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.479087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=400010b050a44efab2072afb4b22edd9, user_id=5883efbb4e634c67b33fab9ec9f3bacc, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.485897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.624222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 844/1692] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:33 2022] PUT /identity/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83/users/7268b8dc2f94404eb132f8cd8f8cec0d/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:33.628058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.642081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 849/1693] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:33 2022] PUT /identity/v3/projects/400010b050a44efab2072afb4b22edd9/users/5883efbb4e634c67b33fab9ec9f3bacc/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:33.648506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:33.768010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:33.797096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a22b-8af8-4e7f-b0ae-a35c70e0fba6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.831578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857fd67-51a0-4fc3-bbbe-8506fdfcc8bd admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:33.862216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.866317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.867828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.868249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.869083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.871096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0421bf-533d-4ae5-bfd5-85f2df9ae06c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:33.905102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 850/1694] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:33.905102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.905102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.905102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.946939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] MFA Rules not processed for user `5f405a8b7cd64c7e9615a528a0d2484b`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:33.948317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:33.982890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:33.985045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:33.988291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:33.990357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:33.993368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:33.998599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:34.029761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 845/1695] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:34.038038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:34.044211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5431e3-d2a0-4630-90eb-81c40a6a33a3 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:34.050960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:34.051640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.052488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.052956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] PATH_INFO: `/v3/projects/400010b050a44efab2072afb4b22edd9/users/5883efbb4e634c67b33fab9ec9f3bacc/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.055594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400010b050a44efab2072afb4b22edd9, user_id=5883efbb4e634c67b33fab9ec9f3bacc, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:34.059001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:34.090658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 846/1696] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:34 2022] PUT /identity/v3/projects/400010b050a44efab2072afb4b22edd9/users/5883efbb4e634c67b33fab9ec9f3bacc/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:34.094881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:34.094881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.159856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 851/1697] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:34.160902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:34.183428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:34.191046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63d127-f07f-439b-8b65-3598825ebd76 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:34.273384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6bc8-ad8a-4a07-a816-d68afd9c9081 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:34.285580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:34.287041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.289585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.290014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] PATH_INFO: `/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.292701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7607d35fe78f4ff6a1d26acdfb036ae6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:34.298222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:34.364139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:34.368055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.368524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.368975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] PATH_INFO: `/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83/users/7268b8dc2f94404eb132f8cd8f8cec0d/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.376181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b881de0f39a4dbf9a8fdfa85bea8b83, user_id=7268b8dc2f94404eb132f8cd8f8cec0d, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:34.379385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6bea5-23c3-43b9-a298-764e0730236a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:34.407511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 852/1698] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:34 2022] PUT /identity/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83/users/7268b8dc2f94404eb132f8cd8f8cec0d/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:34.409356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcdd3-2dcf-4bb4-ba38-d8e603a40c66 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.409636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcdd3-2dcf-4bb4-ba38-d8e603a40c66 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.409906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcdd3-2dcf-4bb4-ba38-d8e603a40c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.490472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-18bbcdd3-2dcf-4bb4-ba38-d8e603a40c66 None None] MFA Rules not processed for user `5883efbb4e634c67b33fab9ec9f3bacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:34.491742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-18bbcdd3-2dcf-4bb4-ba38-d8e603a40c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.495817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7607d35fe78f4ff6a1d26acdfb036ae6', 'request_id': 'req-f86f3bd2-5936-49a8-af4e-df79ca9e8775'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:34.519690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7607d35fe78f4ff6a1d26acdfb036ae6', 'request_id': 'req-f86f3bd2-5936-49a8-af4e-df79ca9e8775'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:34.535729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7607d35fe78f4ff6a1d26acdfb036ae6', 'request_id': 'req-f86f3bd2-5936-49a8-af4e-df79ca9e8775'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:34.625700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] The token cache is being invalidate because project 7607d35fe78f4ff6a1d26acdfb036ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:34.626385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:34.627751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f86f3bd2-5936-49a8-af4e-df79ca9e8775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:34.631646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 847/1699] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:34 2022] DELETE /identity/v3/projects/7607d35fe78f4ff6a1d26acdfb036ae6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:34.633873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e221b-2ccb-449f-982b-8bea03714699 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.633873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e221b-2ccb-449f-982b-8bea03714699 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.633873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e221b-2ccb-449f-982b-8bea03714699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.706957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7f6e221b-2ccb-449f-982b-8bea03714699 None None] MFA Rules not processed for user `5f405a8b7cd64c7e9615a528a0d2484b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:34.706957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7f6e221b-2ccb-449f-982b-8bea03714699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.707174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 853/1700] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:34.708251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.708709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.709248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.736210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:34.766300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] MFA Rules not processed for user `7268b8dc2f94404eb132f8cd8f8cec0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:34.770533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.853881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 848/1701] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:34.857875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:34.857875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:34.857875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:34.894311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:34.896459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.962618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 854/1702] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:34.963687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:34.965767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:34.979407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:35.023836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 849/1703] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:35.024971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.026967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.068585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1150e-bf82-4b16-8f06-8108f22c4b5d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.072839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.073624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.073902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.074168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.100958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.105272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.133269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 855/1704] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:35.134660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.137057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.162363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61af56-ec48-4b2f-8ce8-b436c56c37ef service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.168310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e3be-f263-4782-af9c-9bd8c5110e4d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.171639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.172622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.173441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.174357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.180134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.183425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.183425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.184830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.186185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.190953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.192845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab42131-83c5-49a8-98be-93641c6b86d5 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.203471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.206893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 856/1705] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:35.208254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ce00c-3116-454d-9669-445660a7921f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.208658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ce00c-3116-454d-9669-445660a7921f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.209026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ce00c-3116-454d-9669-445660a7921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.219490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.233875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-177ce00c-3116-454d-9669-445660a7921f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:35.235299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-177ce00c-3116-454d-9669-445660a7921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.250507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 857/1706] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:35.251068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 850/1707] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:35.251963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.255953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.256453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.256809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.273773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdbeb1-3a9e-44a0-bb73-03117536cbc7 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.287002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.287616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.288112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.288463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.289327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:35.289741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.291635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.302577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] MFA Rules not processed for user `528811040ede40bdb40e3e6e666d283d`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:35.303157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.321428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 851/1708] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:15:35.322688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.345806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683cfe-eec5-4784-8f3c-9ad7e9a2a806 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.365275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.366132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.366415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.366747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.368665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.373410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.466284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 858/1709] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:35.507500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 852/1710] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:35 2022] POST /identity/v3/projects => generated 389 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:35.511658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.527295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc0f9d-511f-4b97-bcbe-e7c6327c4d7c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.609958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.633184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:35.646890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.648700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.649261 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.650031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.650979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.654208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.718698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5e9-0209-4f84-9a0e-4cd48d08db7c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.726796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.727463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.727909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.728257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.731688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.751783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 859/1711] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:35 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:35.757376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.775135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5f88-42fb-4c91-a276-1f97c429c8ae admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.787949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:35.789762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:35.790831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:35.791296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:35.792422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.799323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.823024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:35.837069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 860/1712] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:35.837922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.840125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:35.908761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:35.910270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:35.942570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 853/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:35.943982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:35.978549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0044ae3-14db-488f-85d6-2f2f9727345b service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:35.997270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:36.000925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.003209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.003732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.004640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:36.006531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4aae-fe9d-41ea-b828-61692642a7cd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:36.042534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 854/1714] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:36.047408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334a01-6230-47bc-97ae-13db54d3f0b3 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.049301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334a01-6230-47bc-97ae-13db54d3f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.049963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334a01-6230-47bc-97ae-13db54d3f0b3 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.055010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 855/1715] 10.222.0.36 () {54 vars in 912 bytes} [Tue Aug 9 18:15:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:36.056107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58334a01-6230-47bc-97ae-13db54d3f0b3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:36.080615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951b50-12f1-471d-91cf-e19d9f38f4e1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:36.082503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58334a01-6230-47bc-97ae-13db54d3f0b3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:36.087877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:36.092120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:36.092810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.097569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.097569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] PATH_INFO: `/v3/projects/dc6b4c9a830140538f8c3b6b4958f213/users/8af15c3e5e964c87857f561c0c73d8d8/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.097859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.097859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.097859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.131868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6b4c9a830140538f8c3b6b4958f213, user_id=8af15c3e5e964c87857f561c0c73d8d8, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:36.135789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7cad7-bffd-46dc-a6c1-726d96f40041 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:36.220060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 856/1716] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:36 2022] PUT /identity/v3/projects/dc6b4c9a830140538f8c3b6b4958f213/users/8af15c3e5e964c87857f561c0c73d8d8/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:36.221733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.222135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.223499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.252060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:36.253777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf68ce7-d854-4cfe-baba-34547c9a8949 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:36.286797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 861/1717] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:36.289112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.290882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.291791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.358300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:36.359713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:36.403332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] MFA Rules not processed for user `8af15c3e5e964c87857f561c0c73d8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:36.410106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:36.556561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 857/1718] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:36.561999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:36.564243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:36.603304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d0449-b9b3-4aec-bf51-4c67ea36073c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:36.609645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:36.609645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.609645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.609645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.611117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 862/1719] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:36.619545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:36.620267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:36.621513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab97425-5e82-4b2a-b9d5-9aaec96e6e6e service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:36.660136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 858/1720] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:15:36.667462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f733-f6e6-46eb-aa50-d7ece74709c0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.667462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f733-f6e6-46eb-aa50-d7ece74709c0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.667462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f733-f6e6-46eb-aa50-d7ece74709c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.709895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:36.763716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:36.789191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e443f733-f6e6-46eb-aa50-d7ece74709c0 None None] MFA Rules not processed for user `18fae6990e4148ff97dec28b204c52df`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:36.791609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e443f733-f6e6-46eb-aa50-d7ece74709c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:36.827218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230875db-1e61-4697-bf05-6dccd3496bcc None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:36.835412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:36.836561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.836972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.837304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.838109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:36.840844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:36.936034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 859/1721] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:36.950531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:36.951198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:36.951754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:36.997787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 863/1722] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/projects => generated 387 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:36.998895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.033813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] MFA Rules not processed for user `18fae6990e4148ff97dec28b204c52df`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:37.055171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:37.166447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e884ef-4814-4e65-8339-6586da5d2904 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.175658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.177372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.177834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.178244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.179250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.181496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.274110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 860/1723] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:37.302914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 864/1724] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:36 2022] POST /identity/v3/users => generated 300 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:37.311167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.345212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c3621-9c9c-4cbf-8748-d382a33e8144 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.365261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.365261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.365261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.365261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.365595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.365595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.401567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.407284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 861/1725] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:37.420022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.426681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7f41-ca55-4ba6-9d36-ab8c21e02a0e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.443224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.444296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.451541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.451957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] PATH_INFO: `/v3/projects/cab6261bd351480187152cc61f51351f/users/de2065f32d974f39be4cd9b4434daa2e/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.474604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab6261bd351480187152cc61f51351f, user_id=de2065f32d974f39be4cd9b4434daa2e, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.478976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.558740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 865/1726] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:37 2022] PUT /identity/v3/projects/cab6261bd351480187152cc61f51351f/users/de2065f32d974f39be4cd9b4434daa2e/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:37.562225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.563373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80555e-4b45-490c-b559-605cdc4bffcc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.645606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:37.695697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.696845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.696845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.696993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.729038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac135ba-9dea-4d98-bc6c-748b9ca3633d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.750349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.753358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.753951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.754392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.755967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.758649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f17e18-3977-410b-ab1d-7a8aa415cf73 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.769565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 866/1727] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:37.771159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14b83-b7a1-4b36-bab3-5b197aaf2eae None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.771651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14b83-b7a1-4b36-bab3-5b197aaf2eae None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.772863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14b83-b7a1-4b36-bab3-5b197aaf2eae None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.774387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 867/1728] 10.222.0.36 () {54 vars in 961 bytes} [Tue Aug 9 18:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:37.775710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a14b83-b7a1-4b36-bab3-5b197aaf2eae None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.809466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a14b83-b7a1-4b36-bab3-5b197aaf2eae None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:37.824625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:37.825448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.825830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.826208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] PATH_INFO: `/v3/projects/cab6261bd351480187152cc61f51351f/users/de2065f32d974f39be4cd9b4434daa2e/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.827404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.830862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4616146-fdde-4d08-9170-8db678d9e815 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.831798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab6261bd351480187152cc61f51351f, user_id=de2065f32d974f39be4cd9b4434daa2e, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:37.841172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:37.846234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 862/1729] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:15:37.847872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632059-5616-44ec-bcaf-1e0e747acef5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:37.848204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632059-5616-44ec-bcaf-1e0e747acef5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:37.848653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632059-5616-44ec-bcaf-1e0e747acef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:37.868871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:37.877816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a5632059-5616-44ec-bcaf-1e0e747acef5 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:37.879410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a5632059-5616-44ec-bcaf-1e0e747acef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:37.892252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 868/1730] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:37 2022] PUT /identity/v3/projects/cab6261bd351480187152cc61f51351f/users/de2065f32d974f39be4cd9b4434daa2e/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:37.893343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:37.896067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.050663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 863/1731] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Aug 9 18:15:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:38.053673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.053961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.054331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.081927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b320-50e7-4f09-acf5-d640739c14d2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.097702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] MFA Rules not processed for user `de2065f32d974f39be4cd9b4434daa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:38.099941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.105202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.107704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.109462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.110029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] PATH_INFO: `/v3/users/0d9edcdcbae74071bbf54a0904d00d2c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.115841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9edcdcbae74071bbf54a0904d00d2c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.118143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:38.185947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:38.262957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 864/1732] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:38.264642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:38.291913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8703b0-6d08-45ac-a6dc-be4632236dbd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.298041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.298986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.299461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.299933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.311270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.321146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.323201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:38.337631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 865/1733] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:38.339474 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:38.367377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9edcdcbae74071bbf54a0904d00d2c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:38.413258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9edcdcbae74071bbf54a0904d00d2c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:38.418824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9edcdcbae74071bbf54a0904d00d2c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:38.433683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9edcdcbae74071bbf54a0904d00d2c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:38.435241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-81c1c8f1-4637-47b9-adca-49775138f039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9edcdcbae74071bbf54a0904d00d2c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:38.483316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c0a9d-56ac-41a5-bcf6-8f5582a154ba service placement] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.490109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.491690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.492148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.492956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.495130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.523225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.523225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501dd26-c214-4d30-84cc-c15e68cb4ba5 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:38.538734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 866/1734] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:38.540792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06ac00-7805-4bd3-adb4-61b69d60d4e5 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.541094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06ac00-7805-4bd3-adb4-61b69d60d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.541523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06ac00-7805-4bd3-adb4-61b69d60d4e5 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.547810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 867/1735] 10.222.0.36 () {54 vars in 912 bytes} [Tue Aug 9 18:15:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:38.550250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.550539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.551059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.570420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 869/1736] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:37 2022] DELETE /identity/v3/users/0d9edcdcbae74071bbf54a0904d00d2c => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:38.572763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146577f8-22be-4ea6-9108-10335453d21d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.573294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146577f8-22be-4ea6-9108-10335453d21d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.573744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146577f8-22be-4ea6-9108-10335453d21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.577459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:38.578727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.620646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-146577f8-22be-4ea6-9108-10335453d21d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:38.623231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-146577f8-22be-4ea6-9108-10335453d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.727520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 868/1737] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:38.729873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:38.734127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.768367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6424bf-b9a8-4587-838c-0306f2a76633 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.782928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.784162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.784162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.784439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.797201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 870/1738] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:38.805947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.809756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146577f8-22be-4ea6-9108-10335453d21d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:38.811347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-146577f8-22be-4ea6-9108-10335453d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.816590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:38.843926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 869/1739] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:38.850506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146577f8-22be-4ea6-9108-10335453d21d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.857087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:38.859494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:38.865957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.871178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.871178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.871178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.881422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.883720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e9940-f625-44e6-ac6b-5ec1e3952bcf service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:38.889159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496021-54ba-47d4-8704-aa48154785b5 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:38.901742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 871/1740] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:38.918153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:38.918946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:38.920101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:38.921454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] PATH_INFO: `/v3/users/8af15c3e5e964c87857f561c0c73d8d8` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:38.923968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af15c3e5e964c87857f561c0c73d8d8)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:38.929319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:39.154921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af15c3e5e964c87857f561c0c73d8d8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:39.173840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af15c3e5e964c87857f561c0c73d8d8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:39.199722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:39.205362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af15c3e5e964c87857f561c0c73d8d8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:39.302822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af15c3e5e964c87857f561c0c73d8d8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:39.303533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af15c3e5e964c87857f561c0c73d8d8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:39.313061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 870/1741] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:38 2022] DELETE /identity/v3/users/8af15c3e5e964c87857f561c0c73d8d8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:39.318497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:39.318818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:39.319091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:39.367486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:39.386205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:39.389636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:39.561833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 872/1742] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:39.573869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:39.575467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:39.733807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c52c-3e44-42ba-a873-c5c2fe8722c4 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:39.758372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:39.759172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:39.759461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:39.759736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:39.786567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:39.786567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c0715-1d54-475c-aea4-fe4f9dd59721 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:39.877569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 871/1743] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:40.015445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:40.015777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:40.024917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:40.025205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:40.025480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:40.055851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffff1c-a8cb-4032-b55c-7bcd05c4606e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:40.075859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:40.076682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:40.077046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:40.077453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] PATH_INFO: `/v3/projects/a636a52498504f69a9484cbe8d7a1c3a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:40.080231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a636a52498504f69a9484cbe8d7a1c3a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:40.086113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] MFA Rules not processed for user `18fae6990e4148ff97dec28b204c52df`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:40.088077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:40.088202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:40.246614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 872/1744] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:40.380869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a636a52498504f69a9484cbe8d7a1c3a', 'request_id': 'req-ab15fbdf-4177-4d69-8294-1118003c8ca3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:40.398883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a636a52498504f69a9484cbe8d7a1c3a', 'request_id': 'req-ab15fbdf-4177-4d69-8294-1118003c8ca3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:40.410187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a636a52498504f69a9484cbe8d7a1c3a', 'request_id': 'req-ab15fbdf-4177-4d69-8294-1118003c8ca3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:40.498180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] The token cache is being invalidate because project a636a52498504f69a9484cbe8d7a1c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:40.498180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:40.499373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:40.504372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 873/1745] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:40 2022] DELETE /identity/v3/projects/a636a52498504f69a9484cbe8d7a1c3a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:40.506718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:40.526683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca7a7-04d2-4867-8dd3-07eba7a1806a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:40.813557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:40.814729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:40.815052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:40.815322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] PATH_INFO: `/v3/users/5883efbb4e634c67b33fab9ec9f3bacc` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:40.817595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5883efbb4e634c67b33fab9ec9f3bacc)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:40.820763 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:40.936579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:40.946382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:40.993170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15fbdf-4177-4d69-8294-1118003c8ca3 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:41.014955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:41.017755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:41.017916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:41.019884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:41.020781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] PATH_INFO: `/v3/projects/dc6b4c9a830140538f8c3b6b4958f213` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:41.029696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6b4c9a830140538f8c3b6b4958f213)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:41.058182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:41.080327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5883efbb4e634c67b33fab9ec9f3bacc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.111495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5883efbb4e634c67b33fab9ec9f3bacc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.138178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5883efbb4e634c67b33fab9ec9f3bacc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.246552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5883efbb4e634c67b33fab9ec9f3bacc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.250972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-920e8950-56d9-44cd-ab24-a0b3a1e0c1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5883efbb4e634c67b33fab9ec9f3bacc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.265909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 873/1746] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:40 2022] DELETE /identity/v3/users/5883efbb4e634c67b33fab9ec9f3bacc => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:41.282937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6b4c9a830140538f8c3b6b4958f213', 'request_id': 'req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.299598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6b4c9a830140538f8c3b6b4958f213', 'request_id': 'req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.309858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6b4c9a830140538f8c3b6b4958f213', 'request_id': 'req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.475818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:41.476317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:41.476938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:41.486646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] The token cache is being invalidate because project dc6b4c9a830140538f8c3b6b4958f213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:41.486646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.486646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:41.499384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 874/1747] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:40 2022] DELETE /identity/v3/projects/dc6b4c9a830140538f8c3b6b4958f213 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:41.502145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:41.506769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:41.511088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] MFA Rules not processed for user `528811040ede40bdb40e3e6e666d283d`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:41.512561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:41.549708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:41.635687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43112af-20f1-45e0-9d45-1e1a6f10fe9b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:41.671898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 874/1748] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:41.868041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:41.868700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:41.868979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:41.870339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] PATH_INFO: `/v3/users/de2065f32d974f39be4cd9b4434daa2e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:41.873697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2065f32d974f39be4cd9b4434daa2e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:41.875881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:42.147751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:42.148690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:42.193883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b977566-9b83-4b5b-a71e-d1159fdd5fdc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:42.206951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:42.208220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:42.208555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:42.210937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] PATH_INFO: `/v3/projects/400010b050a44efab2072afb4b22edd9` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:42.213808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400010b050a44efab2072afb4b22edd9)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:42.227026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:42.304689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2065f32d974f39be4cd9b4434daa2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.340990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2065f32d974f39be4cd9b4434daa2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.340990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2065f32d974f39be4cd9b4434daa2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.357285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2065f32d974f39be4cd9b4434daa2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.358430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2065f32d974f39be4cd9b4434daa2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.489416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 875/1749] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:41 2022] DELETE /identity/v3/users/de2065f32d974f39be4cd9b4434daa2e => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:42.492610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:42.499708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:42.539776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400010b050a44efab2072afb4b22edd9', 'request_id': 'req-a0a8b807-5346-410d-a705-223daeb6a742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.543132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400010b050a44efab2072afb4b22edd9', 'request_id': 'req-a0a8b807-5346-410d-a705-223daeb6a742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.551033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400010b050a44efab2072afb4b22edd9', 'request_id': 'req-a0a8b807-5346-410d-a705-223daeb6a742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.641204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] The token cache is being invalidate because project 400010b050a44efab2072afb4b22edd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:42.641502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.646547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a0a8b807-5346-410d-a705-223daeb6a742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:42.657778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 875/1750] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:42 2022] DELETE /identity/v3/projects/400010b050a44efab2072afb4b22edd9 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:42.657778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163fb00-8fd8-455a-99ad-98659abb0270 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:42.657778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163fb00-8fd8-455a-99ad-98659abb0270 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:42.657778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163fb00-8fd8-455a-99ad-98659abb0270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:42.759286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d325d3-f2de-4499-85c7-c8a2f5a47a8f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:42.767200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0163fb00-8fd8-455a-99ad-98659abb0270 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:42.774258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0163fb00-8fd8-455a-99ad-98659abb0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:42.782016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:42.782673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:42.783165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:42.783594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] PATH_INFO: `/v3/users/7569e6a68c9d40709954daf6a5e40c08` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:42.795818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7569e6a68c9d40709954daf6a5e40c08)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:42.810981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 876/1751] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:42.813970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:42.824005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:42.827785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:42.828121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:42.922983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:42.930741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:42.969920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 877/1752] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:42.970923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:43.009934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f048c-2c46-4b9b-89d2-1e970a881a57 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:43.059128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:43.059830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:43.063985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:43.064266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:43.068417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:43.079494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:43.082087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:43.124837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 878/1753] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:43.130277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:43.147901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd934ad-903e-444f-aa50-bcaf384917f1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:43.156069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:43.156918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:43.157412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:43.157875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:43.160456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:43.165775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:43.167258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7569e6a68c9d40709954daf6a5e40c08'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:43.193590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7569e6a68c9d40709954daf6a5e40c08'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:43.199146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:43.209091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7569e6a68c9d40709954daf6a5e40c08'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:43.215117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:43.222891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7569e6a68c9d40709954daf6a5e40c08'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:43.226441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f881264f-32e4-49e2-9921-2242dddb7c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7569e6a68c9d40709954daf6a5e40c08'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:43.273625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 879/1754] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:43 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:15:43.277702 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:43.288338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:43.386224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 876/1755] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:42 2022] DELETE /identity/v3/users/7569e6a68c9d40709954daf6a5e40c08 => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:43.390884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:43.394831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:43.394831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:43.498321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:43.507832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:43.581119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890b70-a6bc-45b3-88a6-d42be34ba595 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:43.739110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 877/1756] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:43.753091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:43.760618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:43.826715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:43.827451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:43.828085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:43.828372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:43.831521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:43.839351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:43.919186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db12db-789d-417e-8fcd-58f008686728 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:43.920823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:43.921517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:43.922833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:43.923207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:43.938538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:43.944466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:43.972816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 878/1757] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:43.990835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 880/1758] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:43 2022] POST /identity/v3/users => generated 288 bytes in 714 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:43.995414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:44.015795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee18c-9442-4f3a-aa87-18d774300a1d service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:44.026448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:44.027364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.039671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.041302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.042149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:44.049912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:44.075764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:44.085957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 879/1759] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:44.091988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd945e-16be-40a9-abee-eb8476c99258 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:44.099075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:44.100156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.100378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.101087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.101760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:44.104847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b56a4-e661-456d-8d38-9da415dbc981 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:44.117327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 881/1760] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:44.121117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:44.133910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c3be-c75a-4c3b-8d61-d4339ba98517 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:44.139604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:44.140277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.140719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.140996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] PATH_INFO: `/v3/projects/07ce79e5bbe04114907a30ebda502b83/users/c34d7cb26c7648ea9571256bb604c3f1/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.159359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ce79e5bbe04114907a30ebda502b83, user_id=c34d7cb26c7648ea9571256bb604c3f1, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:44.163294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f191f-3157-4b25-b520-fb89fc3c1d80 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:44.284263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 880/1761] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:44 2022] PUT /identity/v3/projects/07ce79e5bbe04114907a30ebda502b83/users/c34d7cb26c7648ea9571256bb604c3f1/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:44.284905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.285344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.285452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.312837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.313105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.313622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.394313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] MFA Rules not processed for user `c34d7cb26c7648ea9571256bb604c3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:44.396488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:44.499492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:44.502024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:44.576151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 882/1762] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:44.579272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:44.580705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:44.687595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:44.743173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:44.756276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 881/1763] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:15:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:44.757353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:44.796085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0dc0-6c19-4505-9b1f-82422b4bec89 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:44.796303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844c2c8-eba8-4465-8d3d-df8314762f32 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:44.812179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:44.812905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.813187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.813457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] PATH_INFO: `/v3/projects/cab6261bd351480187152cc61f51351f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.815955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab6261bd351480187152cc61f51351f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:44.820613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:44.829934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:44.839340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:44.839779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:44.840057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:44.840769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:44.848934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:44.996119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 882/1764] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:44 2022] POST /identity/v3/projects => generated 375 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:45.000084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:45.126968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab6261bd351480187152cc61f51351f', 'request_id': 'req-647f3147-b5c6-4499-a7dd-96e0574aa65c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.137043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab6261bd351480187152cc61f51351f', 'request_id': 'req-647f3147-b5c6-4499-a7dd-96e0574aa65c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.148817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab6261bd351480187152cc61f51351f', 'request_id': 'req-647f3147-b5c6-4499-a7dd-96e0574aa65c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.170059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df326024-c781-4c7b-86e9-f9204e6fe9ad admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:45.244673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] The token cache is being invalidate because project cab6261bd351480187152cc61f51351f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:45.244968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.245784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.248518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 883/1765] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:44 2022] DELETE /identity/v3/projects/cab6261bd351480187152cc61f51351f => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:45.249567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:45.253189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:45.291649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:45.292048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:45.292489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:45.292914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:45.294003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:45.302366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:45.381160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f3147-b5c6-4499-a7dd-96e0574aa65c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:45.386566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:45.387289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:45.387553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:45.387863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] PATH_INFO: `/v3/projects/aa4976a9e83745b1851277848cc99c37` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:45.389007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4976a9e83745b1851277848cc99c37)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:45.391572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:45.521035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 883/1766] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:44 2022] POST /identity/v3/users => generated 294 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:45.529157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:45.540765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:45.586589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4976a9e83745b1851277848cc99c37', 'request_id': 'req-88a85009-375f-49d0-b4b5-9d3a62bc81d0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.602073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4976a9e83745b1851277848cc99c37', 'request_id': 'req-88a85009-375f-49d0-b4b5-9d3a62bc81d0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.612082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4976a9e83745b1851277848cc99c37', 'request_id': 'req-88a85009-375f-49d0-b4b5-9d3a62bc81d0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.731729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] The token cache is being invalidate because project aa4976a9e83745b1851277848cc99c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:45.732415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.733990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-88a85009-375f-49d0-b4b5-9d3a62bc81d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:45.738755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 884/1767] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:45 2022] DELETE /identity/v3/projects/aa4976a9e83745b1851277848cc99c37 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:45.738755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:45.738755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:45.738755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:45.759040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f1bc-dc2f-44bd-b881-b769cd7a4a8e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:45.772565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:45.773986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:45.876650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:45.904370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:45.907492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:45.907897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:45.914401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:45.914401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:45.960865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 885/1768] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:45.962678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:45.967910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:45.995152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4784b-f240-4f9e-85d4-92a4a359c708 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:46.030480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:46.031527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.032004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.032613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.033480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:46.040794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:46.046860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b39fbf-0b9f-496d-a14e-27e0d549032f service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:46.055879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b8da-2515-4831-a981-be052f03b07f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:46.083992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 884/1769] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:15:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:15:46.086095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.086632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.087301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.102649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 886/1770] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:46.111303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1487-78c9-499b-a2f6-2ea37f565d50 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.111594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1487-78c9-499b-a2f6-2ea37f565d50 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.111862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1487-78c9-499b-a2f6-2ea37f565d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.199950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:46.203372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:46.221406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3c1c1487-78c9-499b-a2f6-2ea37f565d50 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:46.226836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3c1c1487-78c9-499b-a2f6-2ea37f565d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:46.255051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 887/1771] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:46.257520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.258229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.258229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.295568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:46.295568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:46.308315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 885/1772] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:46.309407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:46.360993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bf93a-aedf-4cbe-8209-24be1449732f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:46.378158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:46.378937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.379206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.380159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] PATH_INFO: `/v3/projects/04d559cdab3c430390cee060f20ee42d/users/e79885951bf0441d928da45986572c23/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.389715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:46.466597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d559cdab3c430390cee060f20ee42d, user_id=e79885951bf0441d928da45986572c23, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:46.477234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 888/1773] 10.222.0.84 () {58 vars in 1015 bytes} [Tue Aug 9 18:15:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:46.478302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:46.480349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd55903-767d-4f4f-99b1-a6eb2a66ac4a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:46.570647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc346d23-9170-41e5-8285-7bc3d08c3955 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:46.616813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:46.619098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.619098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.619098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.619098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:46.627001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:46.632724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:46.648139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 886/1774] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:46 2022] PUT /identity/v3/projects/04d559cdab3c430390cee060f20ee42d/users/e79885951bf0441d928da45986572c23/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:46.650084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.650516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.650968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.694025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 889/1775] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:46.694528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:46.697891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:46.730893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9b321-b0cc-4853-b7c7-f15412e775c4 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:46.744756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:46.748250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:46.879346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:46.880399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:46.880765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:46.881091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:46.899554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:46.901024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:46.938543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 890/1776] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:46.939509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:46.999966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 887/1777] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:47.003513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.037056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91555d9-9d98-48b8-9ebe-2768b72510a0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.060797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.061441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.061728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.062003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.062653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.071157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.134449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:47.199639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:47.204391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be98c8-228d-41fa-8e62-84fdc253e51b service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.219189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.220050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.220410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.220791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.222212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.226982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.264816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 888/1778] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:46 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:47.266112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.275113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 891/1779] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:47.276044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.302918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d369ed-6ef6-4c32-bfc1-cab2b324b85c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.467195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.468127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.468576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.469538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.470320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.472477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.487934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bdb50-a330-4827-9ade-f1666ab4a144 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.501191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.502065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.502524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.503607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.505385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:47.505897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.508378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.538570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 889/1780] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:47.538570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.566023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7388-5ef3-45a4-af2f-ba896a8990ed admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.587839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.588682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.589067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.589446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] PATH_INFO: `/v3/projects/04d559cdab3c430390cee060f20ee42d/users/e79885951bf0441d928da45986572c23/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.595359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d559cdab3c430390cee060f20ee42d, user_id=e79885951bf0441d928da45986572c23, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.607935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.836129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 890/1781] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:47 2022] PUT /identity/v3/projects/04d559cdab3c430390cee060f20ee42d/users/e79885951bf0441d928da45986572c23/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:47.836129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.838516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 892/1782] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:47 2022] POST /identity/v3/users => generated 287 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:47.838516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:47.838516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d95a3-3e65-4ba1-830a-ad73229e87e9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.839277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363bfa0-0f02-4628-91aa-3ee571b25687 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:47.839277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.839277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.839277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.840548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.840548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.840548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.840814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:47.840814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.840814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.840814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.844744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:47.851538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606ceaf-36a7-4ab9-8d3f-73b5557d5f8d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:47.900062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 893/1783] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:47.900062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:47.900062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:47.900062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:47.906267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] MFA Rules not processed for user `e79885951bf0441d928da45986572c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:47.907794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:48.101420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 894/1784] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:48.101420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:48.133821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57617bc-5737-449c-8690-caabe9252a90 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:48.143120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:48.143742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:48.144043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:48.144293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:48.145016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:48.146639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:48.158103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 895/1785] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:48.159073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:48.160986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:48.292904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 891/1786] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:47 2022] POST /identity/v3/projects => generated 375 bytes in 651 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:48.293768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:48.334965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b38a9-4855-495a-b7f3-d468022bcfc4 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:48.434733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:48.434733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:48.434733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:48.434733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:48.435699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:48.575788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cb24-5744-4074-b69e-ea678d17dc9f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:48.589650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:48.590806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:48.591361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:48.591777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] PATH_INFO: `/v3/projects/de78293ee7a24236a09e7c44abaea3cc/users/57daef7ef4984d6fb8e50861ce258597/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:48.635247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de78293ee7a24236a09e7c44abaea3cc, user_id=57daef7ef4984d6fb8e50861ce258597, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:48.653432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:48.654895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1cfe-ea14-40fe-895d-88557eba5342 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:48.664859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:48.678063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 896/1787] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:48.679175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:48.696535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40425-afa6-4ee9-b810-a1fdf53b8eae service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:48.705013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:48.710651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:48.710651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:48.710651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:48.710651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:48.711660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:48.778621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 892/1788] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:48 2022] PUT /identity/v3/projects/de78293ee7a24236a09e7c44abaea3cc/users/57daef7ef4984d6fb8e50861ce258597/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:48.792270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:48.792552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:48.792818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:48.867442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 897/1789] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:48 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:48.872349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:48.901086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f72be-5a84-4264-aec5-162c4746b976 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:48.910475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] MFA Rules not processed for user `57daef7ef4984d6fb8e50861ce258597`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:48.918510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:49.010974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:49.011520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.012423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.013054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.014138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:49.017625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:49.047798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 898/1790] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:49.053071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:49.081783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25759151-4d0f-4751-ad92-e8fa396d2005 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:49.089377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:49.104167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:49.105398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.107468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.108198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.109521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:49.117609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:49.148614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 899/1791] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:49.153752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:49.154104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 893/1792] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:49.186829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec27f-6b71-460f-b264-4e4488637676 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:49.198139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:49.201605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.203031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.204753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] PATH_INFO: `/v3/projects/3e6cc8830ee5419a9eca383912f27a2b/users/f7c50adc50bd439a9e6303b19863e7d9/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.297652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6cc8830ee5419a9eca383912f27a2b, user_id=f7c50adc50bd439a9e6303b19863e7d9, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:49.302762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac30d61-7666-41f7-8227-e5f853e5111b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:49.338590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:49.340895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:49.434156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:49.448369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 900/1793] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:49 2022] PUT /identity/v3/projects/3e6cc8830ee5419a9eca383912f27a2b/users/f7c50adc50bd439a9e6303b19863e7d9/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:49.463668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d798-e90d-44bd-b803-0230a7571c48 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.464146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d798-e90d-44bd-b803-0230a7571c48 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.464367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d798-e90d-44bd-b803-0230a7571c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.506598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0ca7d798-e90d-44bd-b803-0230a7571c48 None None] MFA Rules not processed for user `f7c50adc50bd439a9e6303b19863e7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:49.507919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0ca7d798-e90d-44bd-b803-0230a7571c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:49.541757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb003-fd35-410f-b7ba-657443d93cf4 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:49.664904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 901/1794] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:49.710784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:49.714846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.714846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.714846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.743599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb4b4a-d79c-41a7-9056-b96865416fbe None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.743599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb4b4a-d79c-41a7-9056-b96865416fbe None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.744901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb4b4a-d79c-41a7-9056-b96865416fbe None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.747085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 902/1795] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:49.847345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:49.849541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:49.865759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:49.872617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 894/1796] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:49.874693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77e221-ca46-4534-8dbb-61a6ecf916f7 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:49.875294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77e221-ca46-4534-8dbb-61a6ecf916f7 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:49.875294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77e221-ca46-4534-8dbb-61a6ecf916f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:49.907668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:49.923038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3a77e221-ca46-4534-8dbb-61a6ecf916f7 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:49.924269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3a77e221-ca46-4534-8dbb-61a6ecf916f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:50.105913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1a40-d3c4-4f1e-9af7-16ebc0559cd1 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:50.118804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:50.119511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:50.119796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:50.120072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] PATH_INFO: `/v3/users/57daef7ef4984d6fb8e50861ce258597` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:50.121899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57daef7ef4984d6fb8e50861ce258597)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:50.123888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:50.127248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 903/1797] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:15:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:50.250506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:50.339650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57daef7ef4984d6fb8e50861ce258597'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:50.359533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57daef7ef4984d6fb8e50861ce258597'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:50.360495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57daef7ef4984d6fb8e50861ce258597'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:50.376419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57daef7ef4984d6fb8e50861ce258597'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:50.378429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57daef7ef4984d6fb8e50861ce258597'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:50.477840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 895/1798] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:49 2022] DELETE /identity/v3/users/57daef7ef4984d6fb8e50861ce258597 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:50.479857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:50.480713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:50.480842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:50.537677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:50.540351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:50.624296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:50.632163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:50.781734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 904/1799] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:50.808922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:50.814042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:50.833560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd44b2-f48c-4594-9d16-c3df70da0d94 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:50.846762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:50.847432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:50.848332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:50.849647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:50.867648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:50.875888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:50.962170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96651c11-a42f-4b74-b8ec-51abf572246c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:50.974686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:50.979968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:50.980451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:50.981322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:51.006860 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:51.012100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:51.021409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:51.023687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:51.040426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 896/1800] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:15:51.070228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 905/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:51.082928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:51.084662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:51.118837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:51.130546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe77db-5ad1-4aab-908e-ef93bff16060 service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:51.152584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:51.155412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:51.155923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:51.156477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:51.163646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:51.211897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:51.213830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26294fc3-d678-40ad-a0d9-ce653f00f590 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:51.240743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 897/1802] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:51.776333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:51.778192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:51.812595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b098-23e4-4fa0-b02d-56686ea7ebaa service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:51.828001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:51.829795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:51.830145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:51.834636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] PATH_INFO: `/v3/users/f7c50adc50bd439a9e6303b19863e7d9` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:51.839873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c50adc50bd439a9e6303b19863e7d9)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:51.845249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:52.104619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c50adc50bd439a9e6303b19863e7d9'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:52.120107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c50adc50bd439a9e6303b19863e7d9'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:52.127906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:52.137141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c50adc50bd439a9e6303b19863e7d9'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:52.230164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c50adc50bd439a9e6303b19863e7d9'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:52.230164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c50adc50bd439a9e6303b19863e7d9'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:52.250076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e458-ba34-4b6f-ae81-c508c1ec440c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:52.264295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 906/1803] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:15:51 2022] DELETE /identity/v3/users/f7c50adc50bd439a9e6303b19863e7d9 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:52.265069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e458-ba34-4b6f-ae81-c508c1ec440c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:52.265069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e458-ba34-4b6f-ae81-c508c1ec440c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:52.305536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9099e458-ba34-4b6f-ae81-c508c1ec440c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:52.308170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9099e458-ba34-4b6f-ae81-c508c1ec440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:52.361030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:52.564431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 898/1804] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:52.611926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:52.614608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:52.738773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7bd14-0ff7-43ee-abc3-ab198e127c21 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:52.747541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:52.748239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:52.754737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:52.754737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:52.764786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:52.767688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:52.794905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 907/1805] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:53.230177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499ca3-8498-40fb-aef4-7cbb17384fc5 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.230279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499ca3-8498-40fb-aef4-7cbb17384fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.230531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499ca3-8498-40fb-aef4-7cbb17384fc5 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.232839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 899/1806] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:15:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:15:53.237662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:53.239711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.256758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a99b-f867-4a14-99f4-e08a165897c9 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:53.260006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.260497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.260600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.262681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:53.263517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.263995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.264434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] PATH_INFO: `/v3/projects/de78293ee7a24236a09e7c44abaea3cc` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.266279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de78293ee7a24236a09e7c44abaea3cc)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:53.276260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:53.302690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:53.314212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.452568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 900/1807] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:15:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:53.461300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de78293ee7a24236a09e7c44abaea3cc', 'request_id': 'req-ab0b83ae-0309-41c9-8086-9bfafd182ff5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:53.480130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de78293ee7a24236a09e7c44abaea3cc', 'request_id': 'req-ab0b83ae-0309-41c9-8086-9bfafd182ff5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:53.487687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de78293ee7a24236a09e7c44abaea3cc', 'request_id': 'req-ab0b83ae-0309-41c9-8086-9bfafd182ff5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:53.498861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:53.502986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.526222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3618-84ad-45e0-b4e0-0f3bbe9a65f6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:53.585978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] The token cache is being invalidate because project de78293ee7a24236a09e7c44abaea3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:53.586267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:53.588241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ab0b83ae-0309-41c9-8086-9bfafd182ff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:53.592124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 908/1808] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:53 2022] DELETE /identity/v3/projects/de78293ee7a24236a09e7c44abaea3cc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:53.597385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c27a-3268-435d-b020-58d71330a752 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.598491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c27a-3268-435d-b020-58d71330a752 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.598929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c27a-3268-435d-b020-58d71330a752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.657406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:53.658757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.662937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.662937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.662937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.663800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7fe3c27a-3268-435d-b020-58d71330a752 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:53.666218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7fe3c27a-3268-435d-b020-58d71330a752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.678237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:53.680284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:53.722182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 901/1809] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:53.724061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:53.746990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.828415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 909/1810] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:53.836252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.836777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.837420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.857532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:53.865894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:53.871834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:53.878492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 910/1811] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:53.883513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:53.890508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6da7-34de-47db-9ab7-5f3422964a41 service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:53.893133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5cc95-d276-4a22-affb-0fe704fe390b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:53.897624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:53.898268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.898630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.898942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] PATH_INFO: `/v3/projects/3e6cc8830ee5419a9eca383912f27a2b` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.899169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:53.899850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.900067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.900162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6cc8830ee5419a9eca383912f27a2b)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:53.900330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.900973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:53.901268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:53.902103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:53.902895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:53.919235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 911/1812] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:53.920274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:53.931709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e6a52-6e8c-492b-969c-e9f69c80a0d9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:53.937342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:53.937918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:53.938195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:53.938463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:53.939132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:53.941068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:53.998616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:54.041506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6cc8830ee5419a9eca383912f27a2b', 'request_id': 'req-21b82c58-3728-42d9-8f3e-3436c978cf59'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:54.047656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6cc8830ee5419a9eca383912f27a2b', 'request_id': 'req-21b82c58-3728-42d9-8f3e-3436c978cf59'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:54.055863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6cc8830ee5419a9eca383912f27a2b', 'request_id': 'req-21b82c58-3728-42d9-8f3e-3436c978cf59'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:54.071011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 912/1813] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:53 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:54.075774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.152274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] The token cache is being invalidate because project 3e6cc8830ee5419a9eca383912f27a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:15:54.152459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:54.154029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-21b82c58-3728-42d9-8f3e-3436c978cf59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:15:54.155655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 902/1814] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:15:53 2022] DELETE /identity/v3/projects/3e6cc8830ee5419a9eca383912f27a2b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:54.158405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef0a-6ab7-4908-818b-cb0e81063a6c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.161857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2738-f3d7-4cbb-95f6-d8b22a023842 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.162146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2738-f3d7-4cbb-95f6-d8b22a023842 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.163721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2738-f3d7-4cbb-95f6-d8b22a023842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.177493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.177743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.177957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.178234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.178885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.183000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.193330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ba3c2738-f3d7-4cbb-95f6-d8b22a023842 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:54.194600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ba3c2738-f3d7-4cbb-95f6-d8b22a023842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.204157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 903/1815] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:54.213033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.213321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.213597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.234508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:54.235512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.245112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 904/1816] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:54.249550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.259831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086efa5-9a70-4764-aaef-f24a438c3db1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.265652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.266506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.266978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.267383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.268236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:15:54.269657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.274111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.293098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 913/1817] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:54.295022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.298355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.306043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 905/1818] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:15:54.310666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.324526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543f955-279b-4b16-9ce4-9b5acf4ac7e7 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.330984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.331597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.331897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.332823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.333652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.333841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259193e-4adc-4003-9c87-aa57246f8423 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.335996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.345102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.345726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.346035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.346286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.347062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.348827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.359497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 914/1819] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:54.364212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.377545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0734-b6f6-4c87-b3d7-d4eb0f063c04 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.386335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.387703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.388120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.388547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.389346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.391548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.406845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 915/1820] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:54.411141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.412238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 906/1821] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:54.416351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.427648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be765-c928-4955-ab7b-5f1497c15e0d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.522343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92313628-0f82-4287-bd73-da7e058c1078 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.529070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.529770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.530123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.530458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.531202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.533188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.556843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.559564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.559564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.559564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] PATH_INFO: `/v3/projects/e4779b588ed94bf391163d649d43f954/users/32d86783396f4fea83db192628f10679/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.577821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4779b588ed94bf391163d649d43f954, user_id=32d86783396f4fea83db192628f10679, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.581977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884480-8501-4a0a-8151-ce777f4e51c8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.599393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 907/1822] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:54.603562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.614431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7473c7-cfd5-4ea2-a936-0003c2f4f5e2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.619139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.619735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.620005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.620270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.620917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.622555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.623824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:54.633428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 908/1823] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:54.637367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.647379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6215-17ea-4d4b-a4fa-8b9f19acf82f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.650167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 916/1824] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:54 2022] PUT /identity/v3/projects/e4779b588ed94bf391163d649d43f954/users/32d86783396f4fea83db192628f10679/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:54.666894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.666894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.666894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.692591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] MFA Rules not processed for user `32d86783396f4fea83db192628f10679`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:54.693845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.704595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.705199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.705513 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.705777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.706409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.708233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.717957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 909/1825] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:15:54.722036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.734063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad3ec-ef6b-4da4-bd44-19feaf76ab7c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.739921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.740572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.740849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.741124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] PATH_INFO: `/v3/projects/bbf36c16034e4094b2401f5652019ad2/users/23002e7507ce49049471c8bd0e997d32/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.755820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf36c16034e4094b2401f5652019ad2, user_id=23002e7507ce49049471c8bd0e997d32, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.758564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8d46-a4d0-4c64-beda-9c8679a70415 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.761222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 917/1826] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:54.772279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.773708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.826642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 910/1827] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:54 2022] PUT /identity/v3/projects/bbf36c16034e4094b2401f5652019ad2/users/23002e7507ce49049471c8bd0e997d32/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:15:54.830228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8d14-1d0f-4eb7-a01e-ba72c5cdd896 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:54.843071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.843361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.843633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.873940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] MFA Rules not processed for user `23002e7507ce49049471c8bd0e997d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:54.875226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:54.882351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:54.882987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:54.883264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:54.883532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:54.935556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:54.943142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 911/1828] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:15:54.947525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:54.947816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:54.949258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:54.958871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 918/1829] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:15:55.011913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c6c03-7c95-46aa-9f40-bc6c27609e3b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.016910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.017519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.017798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.018067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.018689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.021007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.091924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 912/1830] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:15:54 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:15:55.093190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.104281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664260a-52df-4ebf-93c4-dc1d93ea0228 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.163877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.164486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.164759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.165033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.165655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.167612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.278496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 919/1831] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:15:55 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:15:55.282655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.286824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:55.295214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725497-3c8e-4bbe-b919-5085407736c8 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.299993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.301238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.301614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.301888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.302515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.304123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.314493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 913/1832] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:55.318864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.329059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb28e-e8da-4919-a514-2c0037ee223d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.333797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.334588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.334812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.335094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] PATH_INFO: `/v3/projects/a58ccd2005574c66921fa03b313b25bb/users/71cd0a7676a8448eb979921e94ad15c5/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.353121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58ccd2005574c66921fa03b313b25bb, user_id=71cd0a7676a8448eb979921e94ad15c5, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.356107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.428632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 920/1833] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:55 2022] PUT /identity/v3/projects/a58ccd2005574c66921fa03b313b25bb/users/71cd0a7676a8448eb979921e94ad15c5/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:55.429867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.440850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02c4b-6868-415d-aea9-53d7789b1162 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.490761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:55.509360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.510244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.510447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.510735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.511390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.512996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172035fa-92fc-45df-92b3-163b5491de12 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.524002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 914/1834] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:15:55.528409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.543318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44f166-423c-4327-bf14-014c3e5ddfd7 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.571513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.572870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.576378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.576912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] PATH_INFO: `/v3/projects/a58ccd2005574c66921fa03b313b25bb/users/71cd0a7676a8448eb979921e94ad15c5/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.580764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58ccd2005574c66921fa03b313b25bb, user_id=71cd0a7676a8448eb979921e94ad15c5, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.583894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.610230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.610522 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.611200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.614993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 921/1835] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:15:55 2022] PUT /identity/v3/projects/a58ccd2005574c66921fa03b313b25bb/users/71cd0a7676a8448eb979921e94ad15c5/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:15:55.663208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] MFA Rules not processed for user `71cd0a7676a8448eb979921e94ad15c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:15:55.664748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:55.746411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 915/1836] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:15:55.757918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:55.816508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3de690-6ac7-4cd2-80fc-a579456394d7 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:55.820124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:55.820727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:55.820989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:55.821262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:55.830488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:55.832476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:55.841923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 922/1837] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:15:56.075269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:56.085586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840e47d-e443-4985-a03c-4fc18f24bc92 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:56.089101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:56.089727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:56.090002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:56.090271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:56.147218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:56.148900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:56.158044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 916/1838] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:15:56.186458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:15:58.227611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:15:58.229094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:58.240442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e6aaf-4ff3-40cb-8c1c-6e03394ba345 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:15:58.244179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:15:58.244885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:15:58.245165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:15:58.245430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:15:58.247098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:15:58.314281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:15:58.316526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2bf0-4f45-47c4-ab43-46c2f0325678 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:15:58.328160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 923/1839] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:15:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:06.672456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:07.042123 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:07.130122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1a66c-7b34-4f73-a380-52ef69a9bce1 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:07.130122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:07.130122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:07.130122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:07.130497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] PATH_INFO: `/v3/users/32d86783396f4fea83db192628f10679` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:07.130497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d86783396f4fea83db192628f10679)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:07.131544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:07.377360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d86783396f4fea83db192628f10679'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:07.390055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d86783396f4fea83db192628f10679'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:07.390817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d86783396f4fea83db192628f10679'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:07.398031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d86783396f4fea83db192628f10679'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:07.398680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d86783396f4fea83db192628f10679'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:07.483994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 917/1840] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:07 2022] DELETE /identity/v3/users/32d86783396f4fea83db192628f10679 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:07.489786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:07.490309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:07.490390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:07.505697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:07.512116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:07.513475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:07.585710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 924/1841] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:07.598229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:07.599978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:07.661295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bef19b-ce8a-4e3a-b70a-4221547ed7aa admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:07.694632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:07.694632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:07.694632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:07.694632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:07.695480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:07.695480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a307524-ce9a-4233-8f33-b7193b283f58 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:07.695480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 918/1842] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:08.890474 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:08.894328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:08.911943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06cd2-d838-48f2-a524-e34648633bb3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:08.912955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:08.913593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:08.913867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:08.914134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] PATH_INFO: `/v3/projects/e4779b588ed94bf391163d649d43f954` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:08.915292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4779b588ed94bf391163d649d43f954)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:08.917236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.044968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4779b588ed94bf391163d649d43f954', 'request_id': 'req-b0370946-c114-4319-96ca-38ef89938d15'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.051376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4779b588ed94bf391163d649d43f954', 'request_id': 'req-b0370946-c114-4319-96ca-38ef89938d15'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.057877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4779b588ed94bf391163d649d43f954', 'request_id': 'req-b0370946-c114-4319-96ca-38ef89938d15'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.157906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] The token cache is being invalidate because project e4779b588ed94bf391163d649d43f954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:09.159867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.159867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b0370946-c114-4319-96ca-38ef89938d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.161207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 925/1843] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:08 2022] DELETE /identity/v3/projects/e4779b588ed94bf391163d649d43f954 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:09.167856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.168345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.168432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.188564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:09.194324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:09.219802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:09.263994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 919/1844] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:16:09.273181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.273449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.274523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.294564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:09.295790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:09.305295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 926/1845] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:09.309897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.319712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e564d0-3fa2-4ceb-afba-edad386e16f1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.324782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.326179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.326179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.326179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.326613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:09.326944 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.328623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.340966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 920/1846] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:09.345665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.362553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f7440-05dd-4dc8-9074-5ce0b5138941 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.362553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.362553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.362553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.362884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.363217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.365231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.462598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 927/1847] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:09.467293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.480762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe728fc-d00b-4878-b4b7-9a1b5c115091 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.533319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.535941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:09.553918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.554621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.555053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.555417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.556249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.558352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.611164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d71ab-b4cd-4e4b-877c-5017dc2a160c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.616993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.617647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.618078 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.618346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] PATH_INFO: `/v3/users/cbd99565107047e5a2a392ae7297a1a4` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.619528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd99565107047e5a2a392ae7297a1a4)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.621463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.623222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:09.632652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 921/1848] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:09.637114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.648219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88faa20-529d-4dd9-a368-f2bcedb62c38 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.653680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.654282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.654743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.655020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.655658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.657321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.670562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 922/1849] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:09.674762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.686891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966e6c8-3e04-4713-8230-1c70f4e5f1df admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.692137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.692573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.692870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.693144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.693779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.695428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.708395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 923/1850] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:16:09.720741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:09.736194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22187ccd-f910-4523-8875-f3e91f64a5c3 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:09.745334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:09.746075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.746919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.746919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] PATH_INFO: `/v3/projects/74026e49dc734861b9fd108d303a7b57/users/b37df5d2b38f49d6a0ce62a511c85630/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.762950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=74026e49dc734861b9fd108d303a7b57, user_id=b37df5d2b38f49d6a0ce62a511c85630, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:09.765977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304c715-9ae7-4e7d-964e-247850e1dbbf admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:09.800311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd99565107047e5a2a392ae7297a1a4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.807468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd99565107047e5a2a392ae7297a1a4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.825541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd99565107047e5a2a392ae7297a1a4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.871644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 924/1851] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:09 2022] PUT /identity/v3/projects/74026e49dc734861b9fd108d303a7b57/users/b37df5d2b38f49d6a0ce62a511c85630/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:09.883559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.883850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.884126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.894495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd99565107047e5a2a392ae7297a1a4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.895124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e4903a65-dac1-4b49-bb18-5e02905cc9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd99565107047e5a2a392ae7297a1a4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:09.902216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 928/1852] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:09 2022] DELETE /identity/v3/users/cbd99565107047e5a2a392ae7297a1a4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:16:09.906507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:09.906816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:09.907087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:09.914235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] MFA Rules not processed for user `b37df5d2b38f49d6a0ce62a511c85630`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:09.915532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:09.927477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:09.928704 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:09.961612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:09.985011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 925/1853] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:16:09.989009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 929/1854] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:16:09.999449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:10.001449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:10.001965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:10.002047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:10.067303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89ea3f-d892-4687-8ea1-132515affb3f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:10.071094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:10.071715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:10.071992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:10.072259 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:10.081819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:10.083285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:10.089369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d3b2b-8cb4-4a0a-8e0e-b378c02d3439 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:10.096971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 926/1855] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:10.097081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:10.097081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:10.097081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:10.097081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:10.103515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:10.105138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:10.115553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 930/1856] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:10.531106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:10.532777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:10.549252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f242ce-b61a-48a4-af8f-f6703bef0e4f service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:10.555389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:10.556162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:10.556500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:10.556703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] PATH_INFO: `/v3/projects/e2a41d2c4ff343e59f3ca24bbdae5e41` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:10.557815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a41d2c4ff343e59f3ca24bbdae5e41)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:10.559758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:10.683660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a41d2c4ff343e59f3ca24bbdae5e41', 'request_id': 'req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:10.689362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a41d2c4ff343e59f3ca24bbdae5e41', 'request_id': 'req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:10.696269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a41d2c4ff343e59f3ca24bbdae5e41', 'request_id': 'req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:10.760494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] The token cache is being invalidate because project e2a41d2c4ff343e59f3ca24bbdae5e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:10.760874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:10.763764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:10.764953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 927/1857] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:10 2022] DELETE /identity/v3/projects/e2a41d2c4ff343e59f3ca24bbdae5e41 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:10.769995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:10.782757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:10.790313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2b5f5-ea2b-43ee-9c1b-115f8413d762 service glance] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:10.861172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:10.862235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:10.862577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:10.862961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] PATH_INFO: `/v3/users/7268b8dc2f94404eb132f8cd8f8cec0d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:10.871448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7268b8dc2f94404eb132f8cd8f8cec0d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:10.873953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:11.062555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7268b8dc2f94404eb132f8cd8f8cec0d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.071690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7268b8dc2f94404eb132f8cd8f8cec0d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.086376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7268b8dc2f94404eb132f8cd8f8cec0d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.168144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7268b8dc2f94404eb132f8cd8f8cec0d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.168767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9815c0d0-0959-4eba-8f11-a1d5b945d2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7268b8dc2f94404eb132f8cd8f8cec0d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.175252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 931/1858] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:10 2022] DELETE /identity/v3/users/7268b8dc2f94404eb132f8cd8f8cec0d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:11.639488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:11.640868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:11.662410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8a4c2-9825-42ea-87b6-bf5f1f02fcf2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:11.721337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:11.722559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:11.722880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:11.723217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] PATH_INFO: `/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:11.724425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b881de0f39a4dbf9a8fdfa85bea8b83)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:11.726316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:11.738262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:11.861039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b881de0f39a4dbf9a8fdfa85bea8b83', 'request_id': 'req-22bdade4-6ce8-4092-a8bc-57684f3f8a81'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.866609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b881de0f39a4dbf9a8fdfa85bea8b83', 'request_id': 'req-22bdade4-6ce8-4092-a8bc-57684f3f8a81'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.873509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b881de0f39a4dbf9a8fdfa85bea8b83', 'request_id': 'req-22bdade4-6ce8-4092-a8bc-57684f3f8a81'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.939055 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] The token cache is being invalidate because project 0b881de0f39a4dbf9a8fdfa85bea8b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:11.943420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.943420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22bdade4-6ce8-4092-a8bc-57684f3f8a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:11.943420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 928/1859] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:11 2022] DELETE /identity/v3/projects/0b881de0f39a4dbf9a8fdfa85bea8b83 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:11.947125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:11.947411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:11.947710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:11.968076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:11.969279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:12.044638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 932/1860] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:12.046499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.046817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.047095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.074905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:12.076425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:12.087792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 929/1861] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:12.092158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.102257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f228744-be6f-4622-8d1f-2bc57b3aa3ee None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.106943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.107551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.107824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.108090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.108730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:12.109025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.110687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.123587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 933/1862] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:12.128012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.139653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6525-11b6-4426-8d1f-af6c26eb82f3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.145368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.146009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.146286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.146580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.147249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.149270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.246837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 930/1863] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:12.249964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.260541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7290-5f17-407a-accc-87856ce695bc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.265531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:12.318381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.319534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.319534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.319534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.320132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.322063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.418176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 934/1864] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:12.421264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.430564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc111-82e9-46d9-aca9-06e554025416 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.435329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.435925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.436203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.437866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.437866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.439494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.450864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 931/1865] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:12.460480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.472013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52b51-5a39-4cdd-a16a-4d70d303d5f0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.477315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.477913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.478191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.478458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.479126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.480725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401f95-d0db-4de5-afcb-c2fd21a3926e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.492314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 935/1866] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:12.496438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.506568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598027de-2dff-400b-8171-b6a8af210a9c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.511046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.512561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.512561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.512561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] PATH_INFO: `/v3/projects/48c157eb4c584a178b6f8a5496b4daa5/users/8ec5643383644fd78dda99993bc22b5f/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.526902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c157eb4c584a178b6f8a5496b4daa5, user_id=8ec5643383644fd78dda99993bc22b5f, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.529555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.606848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 932/1867] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:12 2022] PUT /identity/v3/projects/48c157eb4c584a178b6f8a5496b4daa5/users/8ec5643383644fd78dda99993bc22b5f/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:12.619265 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.619755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.621814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.648268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] MFA Rules not processed for user `8ec5643383644fd78dda99993bc22b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:12.650739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:12.714407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 936/1868] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:12.719132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.781558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c70c-9104-45a7-bd93-6f54e623ff22 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.787029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.787635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.788000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.788273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.789017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.790993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:12.883911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 933/1869] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:12.887963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:12.898817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e564462-13b7-4379-9848-2acfb70be82d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:12.954671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:12.955298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:12.955573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:12.955841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:12.956460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:12.958563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.042834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 937/1870] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:12 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:13.046833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.057654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ae1d6-45cf-4fe8-9929-6aad3c402c57 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.062723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.063324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.063601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.063870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.064564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.066899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.078630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 934/1871] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:13.082989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.092854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b0135-9940-41d7-ab87-f1e2ae789211 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.097608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.098258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.098534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.098844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] PATH_INFO: `/v3/projects/44b86f973b454055945f9b800b9ee15c/users/1c758635635646d3afb934cbe1d3c975/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.101013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:13.114645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b86f973b454055945f9b800b9ee15c, user_id=1c758635635646d3afb934cbe1d3c975, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.117430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.162754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 938/1872] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:13 2022] PUT /identity/v3/projects/44b86f973b454055945f9b800b9ee15c/users/1c758635635646d3afb934cbe1d3c975/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:13.166624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.177282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a26e-0cb1-46b9-86c6-deb72117b63a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.181802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:13.234776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.235380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.235674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.235923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.236812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.239179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2a04f-0880-483e-8c09-c5dc2489ba17 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.250100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 935/1873] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:13.254554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.265183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0cf6-db76-40e2-902c-5245cb41deb9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.270681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.271499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.271841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.272274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] PATH_INFO: `/v3/projects/44b86f973b454055945f9b800b9ee15c/users/1c758635635646d3afb934cbe1d3c975/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.274215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b86f973b454055945f9b800b9ee15c, user_id=1c758635635646d3afb934cbe1d3c975, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.277049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.289939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 939/1874] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:13 2022] PUT /identity/v3/projects/44b86f973b454055945f9b800b9ee15c/users/1c758635635646d3afb934cbe1d3c975/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:13.301863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.302028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.302220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.331403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] MFA Rules not processed for user `1c758635635646d3afb934cbe1d3c975`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:13.332770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:13.403636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 936/1875] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:13.415495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.417174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:13.498164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b9f43-3855-42b4-a7e3-572925eb7733 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.502153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.502795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.503068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.503336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.514317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.515788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.525836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 940/1876] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:13.801194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:13.810629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2afb4e-4af6-4df5-a368-77ab4568b146 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:13.814168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:13.814788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:13.815068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:13.815339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:13.872538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:13.874103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f861d3-3719-4d4d-b795-bfc141e05015 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:13.882991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 937/1877] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:16.218229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:16.220663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:16.299571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167ba70-1bfb-406a-9bf2-781aeee38ac5 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:16.305061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:16.305726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:16.306004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:16.306417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] PATH_INFO: `/v3/users/23002e7507ce49049471c8bd0e997d32` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:16.307735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23002e7507ce49049471c8bd0e997d32)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:16.309694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:16.479223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23002e7507ce49049471c8bd0e997d32'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:16.486556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23002e7507ce49049471c8bd0e997d32'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:16.499139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23002e7507ce49049471c8bd0e997d32'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:16.500834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:16.608595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23002e7507ce49049471c8bd0e997d32'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:16.609247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23002e7507ce49049471c8bd0e997d32'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:16.615996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 941/1878] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:16 2022] DELETE /identity/v3/users/23002e7507ce49049471c8bd0e997d32 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:16.620759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:16.621266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:16.627531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:16.640553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:16.641793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:16.666561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:16.700988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 938/1879] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:16.711888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:16.713271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:16.768799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340303c9-2c42-4b92-baa6-b628c2fb7b71 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:16.772990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:16.773587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:16.773861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:16.774127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:16.782915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:16.784337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:16.794286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 942/1880] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:17.187990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:17.189520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:17.209024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d47b21-b0e3-4013-800d-6b3a4e9fc1a5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:17.254634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:17.255342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:17.258392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:17.258698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] PATH_INFO: `/v3/projects/bbf36c16034e4094b2401f5652019ad2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:17.266651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf36c16034e4094b2401f5652019ad2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:17.269077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:17.389929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf36c16034e4094b2401f5652019ad2', 'request_id': 'req-1e60bd71-9c21-431d-b1c3-35263d12c500'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.396869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf36c16034e4094b2401f5652019ad2', 'request_id': 'req-1e60bd71-9c21-431d-b1c3-35263d12c500'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.403366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf36c16034e4094b2401f5652019ad2', 'request_id': 'req-1e60bd71-9c21-431d-b1c3-35263d12c500'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.479384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] The token cache is being invalidate because project bbf36c16034e4094b2401f5652019ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:17.482186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.482186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.482873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 939/1881] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:17 2022] DELETE /identity/v3/projects/bbf36c16034e4094b2401f5652019ad2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:17.486357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:17.496692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a2a06-d9c9-44f8-adb1-afb6c0bb4263 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:17.559169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:17.559792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:17.560114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:17.560388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] PATH_INFO: `/v3/users/71cd0a7676a8448eb979921e94ad15c5` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:17.561591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cd0a7676a8448eb979921e94ad15c5)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:17.563564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:17.729548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cd0a7676a8448eb979921e94ad15c5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.737359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cd0a7676a8448eb979921e94ad15c5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.749094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cd0a7676a8448eb979921e94ad15c5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.813011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cd0a7676a8448eb979921e94ad15c5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.813637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4651259c-a2db-463c-b01e-49f100f49c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cd0a7676a8448eb979921e94ad15c5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:17.820653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 943/1882] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:17 2022] DELETE /identity/v3/users/71cd0a7676a8448eb979921e94ad15c5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:16:18.238789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:18.240813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:18.254001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60bd71-9c21-431d-b1c3-35263d12c500 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:18.318920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:18.319686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.319975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.320246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] PATH_INFO: `/v3/projects/a58ccd2005574c66921fa03b313b25bb` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.321388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58ccd2005574c66921fa03b313b25bb)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:18.323354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:18.472523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58ccd2005574c66921fa03b313b25bb', 'request_id': 'req-34ae5939-139a-4d30-ba41-5e712a34cea0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:18.477661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:18.479875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58ccd2005574c66921fa03b313b25bb', 'request_id': 'req-34ae5939-139a-4d30-ba41-5e712a34cea0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:18.486270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58ccd2005574c66921fa03b313b25bb', 'request_id': 'req-34ae5939-139a-4d30-ba41-5e712a34cea0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:18.579533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] The token cache is being invalidate because project a58ccd2005574c66921fa03b313b25bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:18.579872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:18.580571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-34ae5939-139a-4d30-ba41-5e712a34cea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:18.582194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 940/1883] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:18 2022] DELETE /identity/v3/projects/a58ccd2005574c66921fa03b313b25bb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:18.587395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.587915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.587915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.608354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:18.609553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:18.619258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:18.674258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 944/1884] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:18.686598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.686923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.687183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.707101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:18.708358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:18.717258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 941/1885] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:18.728136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:18.737901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab075b-5e59-4597-80fc-859833897fde None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:18.743664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:18.744201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.744475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.744763 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.745406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:18.745732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:18.747372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:18.759051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 945/1886] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:18.763599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:18.773212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c5585-9bb6-45af-b569-20794cbbdd54 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:18.777796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:18.778421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.778723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.778998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.779627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:18.781515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:18.895654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 942/1887] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:18 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:18.900543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:18.910400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca28120-5c06-450a-b236-ac2222ab4646 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:18.965126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:18.965718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:18.966051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:18.966337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:18.966986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:18.968884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.048508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 946/1888] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:18 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:19.052771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.066977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465880ac-78af-4050-a1e3-34b4641c19c7 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.072329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.073040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.073332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.073607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.074588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.076108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.093581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.099576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 943/1889] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:19.112380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b21f6-b3c9-464c-b0fe-ba13bd9bcc6f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.117700 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.118317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.118596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.118920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.119570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.121665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c67625-e9d6-4769-8e54-8577c1ad90e5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.140770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 947/1890] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:19.145434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.160509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce711-c5a4-4232-adb4-a75c1e1ce86a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.167330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.167958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.168243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.168532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] PATH_INFO: `/v3/projects/58e245314f8f4d1fb1d72075bd6111ad/users/8ef2aca9c3fd43c090c9ad4ef3c2854b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.189583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e245314f8f4d1fb1d72075bd6111ad, user_id=8ef2aca9c3fd43c090c9ad4ef3c2854b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.192562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.297205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 944/1891] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:19 2022] PUT /identity/v3/projects/58e245314f8f4d1fb1d72075bd6111ad/users/8ef2aca9c3fd43c090c9ad4ef3c2854b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:19.310779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615fd-b352-45f6-87b1-1a1ca6304a86 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.311022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615fd-b352-45f6-87b1-1a1ca6304a86 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.311304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287615fd-b352-45f6-87b1-1a1ca6304a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.342588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-287615fd-b352-45f6-87b1-1a1ca6304a86 None None] MFA Rules not processed for user `8ef2aca9c3fd43c090c9ad4ef3c2854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:19.343901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-287615fd-b352-45f6-87b1-1a1ca6304a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:19.426791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 948/1892] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:19.428845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.522885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013186f-1053-4160-bd27-4a90c7df0c6a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.541547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.542181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.542464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.543056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.543427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.545527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.657482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507cba-e007-4f07-9c5d-752750556648 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.658046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507cba-e007-4f07-9c5d-752750556648 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.658481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507cba-e007-4f07-9c5d-752750556648 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.660270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 949/1893] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:16:19.671279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.671670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.672042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.700345 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:19.700624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 945/1894] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:19 2022] POST /identity/v3/projects => generated 381 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:16:19.701844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:19.704540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.707418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:19.819773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1844ea-a42d-408f-b8d0-a825d56eac16 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.824514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 950/1895] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:16:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:19.826176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.826817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.827107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.827380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.828006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.830018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.839129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.840456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:19.841370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:19.854659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d9a6d-8f52-4e9f-8f23-fd4a023b86bb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.860800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.861667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.861946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.862237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.871938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.873537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.884974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 951/1896] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:16:19.911515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 946/1897] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:19 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:16:19.916625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.927554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44365730-4206-4d85-baff-9a2e438fce88 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.932373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.932969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.933245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.933515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.934169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.935818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:19.946242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 952/1898] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:19.954109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:19.964979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bacef-1021-4228-afa2-1b3519f3bb94 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:19.969681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:19.970336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:19.970616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:19.970915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] PATH_INFO: `/v3/projects/603cfc6d08dd41f5b6649f30bee58584/users/dd666bd5afd546559a07d85bf754b92b/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:19.985816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=603cfc6d08dd41f5b6649f30bee58584, user_id=dd666bd5afd546559a07d85bf754b92b, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:19.988794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:20.053946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 947/1899] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:19 2022] PUT /identity/v3/projects/603cfc6d08dd41f5b6649f30bee58584/users/dd666bd5afd546559a07d85bf754b92b/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:20.057951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:20.068875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a6f90-32c5-4674-8e28-e7b3b2d15211 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:20.126386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:20.127019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:20.127298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:20.127571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:20.128814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:20.130430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ac12d-4bb0-490e-b463-20e2b6adc369 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:20.142726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 953/1900] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:20.147501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:20.157887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2b73a-5749-416b-9a74-1d40467b6eea admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:20.163600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:20.163600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:20.163881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:20.164154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] PATH_INFO: `/v3/projects/603cfc6d08dd41f5b6649f30bee58584/users/dd666bd5afd546559a07d85bf754b92b/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:20.166068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=603cfc6d08dd41f5b6649f30bee58584, user_id=dd666bd5afd546559a07d85bf754b92b, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:20.168740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:20.179046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 948/1901] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:20 2022] PUT /identity/v3/projects/603cfc6d08dd41f5b6649f30bee58584/users/dd666bd5afd546559a07d85bf754b92b/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:20.189187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:20.189481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:20.189755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:20.218228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] MFA Rules not processed for user `dd666bd5afd546559a07d85bf754b92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:20.219450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:20.283548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:20.288878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 954/1902] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:20.300266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:20.301917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:20.358945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53c0db-0c1c-4256-9c26-425d2e5209cf admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:20.363026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:20.363622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:20.363948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:20.364191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:20.373775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:20.375226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3d1a5-439c-42ae-869d-95ea38d0f988 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:20.385145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 949/1903] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:21.016031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:21.083079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01f6b7-487d-43be-9b9b-9ab7a27a61a6 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:21.088105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:21.088758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:21.089031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:21.089294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] PATH_INFO: `/v3/users/8ef2aca9c3fd43c090c9ad4ef3c2854b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:21.090407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef2aca9c3fd43c090c9ad4ef3c2854b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:21.092422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:21.256958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef2aca9c3fd43c090c9ad4ef3c2854b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:21.264944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef2aca9c3fd43c090c9ad4ef3c2854b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:21.278817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef2aca9c3fd43c090c9ad4ef3c2854b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:21.351855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef2aca9c3fd43c090c9ad4ef3c2854b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:21.352557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef2aca9c3fd43c090c9ad4ef3c2854b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:21.359640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 955/1904] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:21 2022] DELETE /identity/v3/users/8ef2aca9c3fd43c090c9ad4ef3c2854b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:21.364549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:21.364997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:21.365118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:21.384225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:21.385419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:21.447661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 950/1905] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:21.457344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:21.458840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:21.522070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784140ee-66a3-4de7-b0fb-de04522863cc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:21.526208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:21.528696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:21.528696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:21.528696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:21.537170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:21.539081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:21.548825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 956/1906] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:21.999580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:22.001369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:22.020664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba31984-7c23-4ccb-82ba-3f3baf48252a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:22.026254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:22.026934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:22.027209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:22.027640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] PATH_INFO: `/v3/projects/58e245314f8f4d1fb1d72075bd6111ad` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:22.029605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e245314f8f4d1fb1d72075bd6111ad)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:22.031994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:22.152175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e245314f8f4d1fb1d72075bd6111ad', 'request_id': 'req-b0bf2676-b4e5-418f-bcde-c30ab98a9181'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.156473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:22.159129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e245314f8f4d1fb1d72075bd6111ad', 'request_id': 'req-b0bf2676-b4e5-418f-bcde-c30ab98a9181'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.167353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e245314f8f4d1fb1d72075bd6111ad', 'request_id': 'req-b0bf2676-b4e5-418f-bcde-c30ab98a9181'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.247168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] The token cache is being invalidate because project 58e245314f8f4d1fb1d72075bd6111ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:22.247561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.248376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.250064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 951/1907] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:21 2022] DELETE /identity/v3/projects/58e245314f8f4d1fb1d72075bd6111ad => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:22.254041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:22.264166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d97e7c-7b7e-417c-89c5-310e26853011 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:22.285528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:22.334735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:22.335379 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:22.335705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:22.335976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] PATH_INFO: `/v3/users/dd666bd5afd546559a07d85bf754b92b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:22.337124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd666bd5afd546559a07d85bf754b92b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:22.339263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:22.542957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd666bd5afd546559a07d85bf754b92b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.549594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd666bd5afd546559a07d85bf754b92b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.560947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd666bd5afd546559a07d85bf754b92b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.672097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd666bd5afd546559a07d85bf754b92b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.672766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-34e93878-1f57-424b-afd2-09f2adbf3666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd666bd5afd546559a07d85bf754b92b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:22.679975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 957/1908] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:22 2022] DELETE /identity/v3/users/dd666bd5afd546559a07d85bf754b92b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:23.147344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.148776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:23.161829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2676-b4e5-418f-bcde-c30ab98a9181 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.223124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.223761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.224087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.224518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] PATH_INFO: `/v3/projects/603cfc6d08dd41f5b6649f30bee58584` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.225662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=603cfc6d08dd41f5b6649f30bee58584)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.228340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.348040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603cfc6d08dd41f5b6649f30bee58584', 'request_id': 'req-469b6721-9d84-4a92-af81-be33c7db0863'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:23.355024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603cfc6d08dd41f5b6649f30bee58584', 'request_id': 'req-469b6721-9d84-4a92-af81-be33c7db0863'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:23.361688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603cfc6d08dd41f5b6649f30bee58584', 'request_id': 'req-469b6721-9d84-4a92-af81-be33c7db0863'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:23.448316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] The token cache is being invalidate because project 603cfc6d08dd41f5b6649f30bee58584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:23.448745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:23.449296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-469b6721-9d84-4a92-af81-be33c7db0863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:23.451139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 952/1909] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:23 2022] DELETE /identity/v3/projects/603cfc6d08dd41f5b6649f30bee58584 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:23.456231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.457128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.457714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.492526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:23.494641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:23.569567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 958/1910] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:23.576440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.576730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.577011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.598820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:23.600042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:23.608774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 953/1911] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:23.613277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.623516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de89ec6-c3fa-4ff4-839f-3e8f5b57e226 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.628376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.628865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.629140 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.629411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.630341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:23.630341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.632006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.650904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 959/1912] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:23.654872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.664709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08949ebf-15d5-4f00-918b-3b1091b54b4b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.671734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.672417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.672703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.672976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.673611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.678698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.757912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 954/1913] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:23 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:23.762297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.772412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f9d3-42a8-4b92-b4e1-6af0dab8bab2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.797262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:23.830813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.831425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.831708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.831980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.832611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.834618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.907165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 960/1914] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:23 2022] POST /identity/v3/users => generated 281 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:23.916821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.927191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb2e5-c379-4740-bfc7-73a58c013210 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.932440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.933086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.933347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.933657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.934270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.935913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.947229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 955/1915] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:23.951157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.961129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f81d05-7f75-4595-8d63-844ebe25db93 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:23.966376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:23.967069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:23.967349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:23.967623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:23.968262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:23.969859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9008aa-f1cb-44a7-9595-1b3dca691bad admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:23.980659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 961/1916] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:23.984960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:23.996011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82629bab-c5c8-487c-b338-61f96c9c3970 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:24.000591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:24.001237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.001518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.001787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] PATH_INFO: `/v3/projects/a19c018280e04135a899fc8254893528/users/681a5943dcfc4158a82be4624564891e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.016562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19c018280e04135a899fc8254893528, user_id=681a5943dcfc4158a82be4624564891e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:24.019254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:24.083667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 956/1917] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:23 2022] PUT /identity/v3/projects/a19c018280e04135a899fc8254893528/users/681a5943dcfc4158a82be4624564891e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:24.095113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.095401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.095698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.127902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] MFA Rules not processed for user `681a5943dcfc4158a82be4624564891e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:24.129005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:24.194437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 962/1918] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:24.228662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:24.230511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:24.241948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:24.288480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad243ad6-f469-43d4-8018-bf93a5414de2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:24.292187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:24.292788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.293071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.293337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.301660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:24.303143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c99175-4f10-4689-aa3e-6f8b6bc0c6f8 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:24.312313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 957/1919] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:24.328588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:24.394730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a60b8-d6e1-4061-ac6e-1a24d162a291 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:24.399320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:24.400077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.400344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.400610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] PATH_INFO: `/v3/users/681a5943dcfc4158a82be4624564891e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.401732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] RBAC: Authorizing `identity:delete_user(user_id=681a5943dcfc4158a82be4624564891e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:24.403685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:24.612240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:24.621445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681a5943dcfc4158a82be4624564891e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:24.628660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681a5943dcfc4158a82be4624564891e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:24.640876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681a5943dcfc4158a82be4624564891e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:24.716819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681a5943dcfc4158a82be4624564891e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:24.716819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681a5943dcfc4158a82be4624564891e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:24.740412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 963/1920] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:24 2022] DELETE /identity/v3/users/681a5943dcfc4158a82be4624564891e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:24.747948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.748251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.748506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.772322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:24.773687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:24.839438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 958/1921] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:24.849085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:24.850569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:24.921023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7ae1-cb53-4b4f-9f1b-abab4513489b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:24.925057 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:24.925662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:24.925937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:24.926209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:24.936308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:24.937725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b8aa-f4f3-4ead-b503-4c8e7fcb8fa7 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:24.947970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 964/1922] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:25.317446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:25.318981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:25.334969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0c10a-0de9-4ea8-84fd-5140beb43a12 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:25.340370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:25.341248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.341642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.342027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] PATH_INFO: `/v3/projects/a19c018280e04135a899fc8254893528` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.343455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19c018280e04135a899fc8254893528)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:25.345575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:25.454910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19c018280e04135a899fc8254893528', 'request_id': 'req-4f733b69-7fa9-47d8-9e6e-297cc5adf382'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:25.460485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19c018280e04135a899fc8254893528', 'request_id': 'req-4f733b69-7fa9-47d8-9e6e-297cc5adf382'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:25.468223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19c018280e04135a899fc8254893528', 'request_id': 'req-4f733b69-7fa9-47d8-9e6e-297cc5adf382'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:25.540298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] The token cache is being invalidate because project a19c018280e04135a899fc8254893528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:25.541383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:25.542026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4f733b69-7fa9-47d8-9e6e-297cc5adf382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:25.543762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 959/1923] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:25 2022] DELETE /identity/v3/projects/a19c018280e04135a899fc8254893528 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:25.548690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.548966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.549233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.570755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:25.572675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:25.636524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 965/1924] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:25.644661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.652199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.652199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.666234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:25.667444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:25.675317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 960/1925] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:25.679497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:25.690505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7176-fb9a-42e7-ab28-676c0341cffe None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:25.695630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:25.696283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.696555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.696978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.697677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:25.697973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:25.699603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:25.712260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 966/1926] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:25.717872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:25.728084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5be0e-2e12-4b92-b337-303b7fa2051f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:25.732861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:25.734844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.734844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.734844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.734844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:25.736820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:25.833176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 961/1927] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:25 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:25.834898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:25.845728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e2444-b06a-45f7-a76f-e63b48de1900 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:25.900590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:25.912915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:25.913534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:25.914027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:25.914251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:25.915422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:25.919431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:25.983343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 967/1928] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:25 2022] POST /identity/v3/users => generated 282 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:25.985465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:25.996853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410b0df-9f19-4e37-b62e-d3993f95f620 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:26.001545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:26.002465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:26.002799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:26.003090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:26.003815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:26.005428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:26.007111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:26.020356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 962/1929] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:26.027891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:26.048824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f969-b966-402d-885b-dd3f6b0f31de admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:26.053492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:26.054107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:26.054380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:26.057621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:26.057621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:26.058063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52b2a-97b2-4554-97e6-dbda6270546a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:26.074569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 968/1930] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:26.078807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:26.089327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f73bb-8ab5-4152-b10d-953a24ca5e5e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:26.095390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:26.096050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:26.096325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:26.096600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] PATH_INFO: `/v3/projects/be28bcb0619e4ee78aea88b5123d7a2f/users/a400175f70d0495bb2222f51b31279c0/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:26.113700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be28bcb0619e4ee78aea88b5123d7a2f, user_id=a400175f70d0495bb2222f51b31279c0, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:26.116742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:26.186389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 963/1931] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:26 2022] PUT /identity/v3/projects/be28bcb0619e4ee78aea88b5123d7a2f/users/a400175f70d0495bb2222f51b31279c0/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:16:26.197448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:26.197728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:26.198065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:26.234106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] MFA Rules not processed for user `a400175f70d0495bb2222f51b31279c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:26.235250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:26.299667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 969/1932] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:26.311643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:26.313382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:26.371646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22e7d3-33cc-4861-b881-cd29e918253a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:26.375285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:26.375883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:26.376160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:26.376427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:26.385101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:26.386524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1c6c6-e35e-4095-a86c-721e0b34082f service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:26.395242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 964/1933] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:36.296868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:36.391183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:56.001422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:56.002882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:56.068911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9670-7750-44b2-ac1d-c28efc1a3de4 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:56.073489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:56.074115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:56.074393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:56.074685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] PATH_INFO: `/v3/users/8ec5643383644fd78dda99993bc22b5f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:56.075943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec5643383644fd78dda99993bc22b5f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:56.077864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:56.353354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec5643383644fd78dda99993bc22b5f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:56.361741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec5643383644fd78dda99993bc22b5f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:56.376066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec5643383644fd78dda99993bc22b5f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:56.452135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec5643383644fd78dda99993bc22b5f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:56.453033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec5643383644fd78dda99993bc22b5f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:56.471922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 970/1934] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:56 2022] DELETE /identity/v3/users/8ec5643383644fd78dda99993bc22b5f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:56.476348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:56.476816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:56.477043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:56.497284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:56.498538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:56.589565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 965/1935] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:16:56.597374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:56.607805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:56.676331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b8ec-57de-4495-810d-311933051fdc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:56.680449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:56.687196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:56.687196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:56.687196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:56.693357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:56.695038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:56.706923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 971/1936] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:16:57.157629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:57.159177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:57.176511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67261bbf-b5c1-479c-b587-db47c11b0b52 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:57.185570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:57.186186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:57.186462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:57.186768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] PATH_INFO: `/v3/projects/48c157eb4c584a178b6f8a5496b4daa5` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:57.188535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c157eb4c584a178b6f8a5496b4daa5)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:57.190660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:57.323375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c157eb4c584a178b6f8a5496b4daa5', 'request_id': 'req-2d6c0f6a-9467-4388-acbf-3057cfb469e0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.329323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c157eb4c584a178b6f8a5496b4daa5', 'request_id': 'req-2d6c0f6a-9467-4388-acbf-3057cfb469e0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.335821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c157eb4c584a178b6f8a5496b4daa5', 'request_id': 'req-2d6c0f6a-9467-4388-acbf-3057cfb469e0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.419991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] The token cache is being invalidate because project 48c157eb4c584a178b6f8a5496b4daa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:57.420275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.420921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.422458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 966/1937] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:57 2022] DELETE /identity/v3/projects/48c157eb4c584a178b6f8a5496b4daa5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:57.426459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:57.440862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aece3-43db-4dbb-8e01-fe1fec2a922d service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:57.468753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:57.514376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:57.515049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:57.515325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:57.515594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] PATH_INFO: `/v3/users/1c758635635646d3afb934cbe1d3c975` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:57.516728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c758635635646d3afb934cbe1d3c975)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:57.518861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:57.733491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c758635635646d3afb934cbe1d3c975'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.741586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c758635635646d3afb934cbe1d3c975'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.758005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c758635635646d3afb934cbe1d3c975'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.864484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c758635635646d3afb934cbe1d3c975'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.865177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a8cb5027-b266-44f1-8def-5a499c6db296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c758635635646d3afb934cbe1d3c975'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:57.871754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 972/1938] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:16:57 2022] DELETE /identity/v3/users/1c758635635646d3afb934cbe1d3c975 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:58.311602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:58.313192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:58.331979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c0f6a-9467-4388-acbf-3057cfb469e0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:58.391981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:58.392654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.393535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.393813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] PATH_INFO: `/v3/projects/44b86f973b454055945f9b800b9ee15c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.399514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b86f973b454055945f9b800b9ee15c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:58.399514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:58.483461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:58.504877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b86f973b454055945f9b800b9ee15c', 'request_id': 'req-59dec669-d166-46bd-ac49-1140fab4e879'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:58.514904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b86f973b454055945f9b800b9ee15c', 'request_id': 'req-59dec669-d166-46bd-ac49-1140fab4e879'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:58.524143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b86f973b454055945f9b800b9ee15c', 'request_id': 'req-59dec669-d166-46bd-ac49-1140fab4e879'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:58.607546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] The token cache is being invalidate because project 44b86f973b454055945f9b800b9ee15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:16:58.607841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:58.608934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-59dec669-d166-46bd-ac49-1140fab4e879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:16:58.610281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 967/1939] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:16:58 2022] DELETE /identity/v3/projects/44b86f973b454055945f9b800b9ee15c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:58.615211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.615698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.615896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.636151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:58.642428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:58.701247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 973/1940] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:58.708959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.709247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.709515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.733149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:58.733551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:58.741622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 968/1941] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:58.745942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:58.757420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54c5a5-7477-4d7f-bc84-7b4aa9de375c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:58.763323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:58.764008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.764204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.764485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.765122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:16:58.765423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:58.767281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:58.780492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 974/1942] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:16:58.784825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:58.795374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d51424-5a02-40d5-b0a2-5c2ec6e98633 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:58.800557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:58.801162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.801437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.801705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.802323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:58.804272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:58.893958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 969/1943] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:58 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:58.898030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:58.909236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc36eb3-35f4-40ea-8da0-bd1c8fa0fef7 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:58.936596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:58.976376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:58.976983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:58.977255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:58.983477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:58.983477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:58.983477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.070571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 975/1944] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:58 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:59.075529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.085872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403c989-c20a-4c35-88f9-d4182f3e1a53 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.091216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.091926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.092273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.092608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.093263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.094962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.105252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 970/1945] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:59.110309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.120303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeed1-bb11-4289-8408-4fa8bbef225b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.125238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.125832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.126104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.126369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.127036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.128621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7a042-cf39-4469-a900-2f4082c673a4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.142284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 976/1946] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:59.144135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.155221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52491cb8-5e91-4688-acd4-ae1fa4c9213f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.160500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.161152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.161432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.161705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] PATH_INFO: `/v3/projects/8f59f3c8410842e1b61a08a5d8b7c292/users/05b2f08ff084435f9ce61e92fe09e72d/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.177288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f59f3c8410842e1b61a08a5d8b7c292, user_id=05b2f08ff084435f9ce61e92fe09e72d, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.180087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.264650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 971/1947] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:59 2022] PUT /identity/v3/projects/8f59f3c8410842e1b61a08a5d8b7c292/users/05b2f08ff084435f9ce61e92fe09e72d/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:59.275913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.276197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.276468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.307155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] MFA Rules not processed for user `05b2f08ff084435f9ce61e92fe09e72d`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:16:59.308369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:16:59.374298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 977/1948] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:16:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:16:59.378757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.439810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d5f3c-552e-433b-b5ba-ac8b40d8e2cc admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.444274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.444872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.445176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.445441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.446047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.448035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.567604 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 972/1949] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:16:59 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:59.571606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.583298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55994a-90c2-4035-b359-026e46e3eb48 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.645337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.646028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.646210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.646450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.647102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.649009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.756271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 978/1950] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:16:59 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:16:59.760465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.772375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3199a-034c-46e7-8aba-51a0bea8603c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.777078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.778285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.778652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.778956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.779596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.781174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.794994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 973/1951] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:16:59.796178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.806468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fd95-0ba1-4565-961d-4ca34ae9f7f9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.812033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.812768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.813050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.813328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] PATH_INFO: `/v3/projects/694d1961124c42b9a410923c2f4bcf67/users/9bd2cb266fba463a9157cc2cdcb98022/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.829157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=694d1961124c42b9a410923c2f4bcf67, user_id=9bd2cb266fba463a9157cc2cdcb98022, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.831884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:16:59.914356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 979/1952] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:16:59 2022] PUT /identity/v3/projects/694d1961124c42b9a410923c2f4bcf67/users/9bd2cb266fba463a9157cc2cdcb98022/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:16:59.918312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:16:59.928779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207709-1a15-495c-8ec5-031d105ba067 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:16:59.949169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:16:59.985997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:16:59.986593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:16:59.986906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:16:59.987180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:16:59.987807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:16:59.989387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba16892-b313-4913-8821-e2f8fbaf5be0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:00.000092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 974/1953] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:00.007192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:00.013785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:00.019114 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79ca2-5a36-449d-8fd5-5bb542a986ad admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:00.023723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:00.024367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:00.024642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:00.024910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] PATH_INFO: `/v3/projects/694d1961124c42b9a410923c2f4bcf67/users/9bd2cb266fba463a9157cc2cdcb98022/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:00.026924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694d1961124c42b9a410923c2f4bcf67, user_id=9bd2cb266fba463a9157cc2cdcb98022, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:00.032149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:00.042633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 980/1954] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:00 2022] PUT /identity/v3/projects/694d1961124c42b9a410923c2f4bcf67/users/9bd2cb266fba463a9157cc2cdcb98022/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:00.053167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:00.053450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:00.053720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:00.084387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] MFA Rules not processed for user `9bd2cb266fba463a9157cc2cdcb98022`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:00.086310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:00.153715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 975/1955] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:00.165154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:00.166836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:00.224183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddddd0c-cc92-4f75-83f6-24f1caab33c2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:00.227660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:00.228261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:00.228535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:00.228819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:00.239464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:00.240885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:00.251017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 981/1956] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:00.416968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:00.427089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f0a05-736f-4d6a-b443-fc6c9d5a9756 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:00.430575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:00.431192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:00.431469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:00.431736 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:00.491132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:00.492706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70d0d7-2c4f-4c06-94db-643b13925ae6 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:00.502676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 976/1957] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:00.983177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:01.051920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280de73-f458-4245-b21d-0b2dbba9926a service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:01.056967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:01.057621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:01.057915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:01.058207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] PATH_INFO: `/v3/users/05b2f08ff084435f9ce61e92fe09e72d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:01.059655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b2f08ff084435f9ce61e92fe09e72d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:01.061742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:01.198776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b2f08ff084435f9ce61e92fe09e72d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:01.206024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b2f08ff084435f9ce61e92fe09e72d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:01.218447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b2f08ff084435f9ce61e92fe09e72d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:01.264346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b2f08ff084435f9ce61e92fe09e72d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:01.265211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b2f08ff084435f9ce61e92fe09e72d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:01.271889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 982/1958] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:00 2022] DELETE /identity/v3/users/05b2f08ff084435f9ce61e92fe09e72d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:01.276211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:01.276489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:01.276946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:01.298514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:01.300025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:01.340435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:01.359575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 977/1959] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:01.369284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:01.370792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:01.384934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:01.429076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66c928-4690-4131-b75c-79031f53efb2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:01.432972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:01.433417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:01.433695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:01.434082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:01.443573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:01.444996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:01.460359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 983/1960] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:01.928592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:01.930486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:01.944914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303a34-0cde-44d0-b2ec-e178b6d3c6d4 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:01.950199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:01.951130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:01.951520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:01.951886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] PATH_INFO: `/v3/projects/8f59f3c8410842e1b61a08a5d8b7c292` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:01.953914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f59f3c8410842e1b61a08a5d8b7c292)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:01.956963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:02.075562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f59f3c8410842e1b61a08a5d8b7c292', 'request_id': 'req-6b7e013d-85de-43c3-816c-17a66edfb223'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.081545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f59f3c8410842e1b61a08a5d8b7c292', 'request_id': 'req-6b7e013d-85de-43c3-816c-17a66edfb223'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.087855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f59f3c8410842e1b61a08a5d8b7c292', 'request_id': 'req-6b7e013d-85de-43c3-816c-17a66edfb223'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.166563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] The token cache is being invalidate because project 8f59f3c8410842e1b61a08a5d8b7c292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:02.166805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.167567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.169424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 978/1961] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:01 2022] DELETE /identity/v3/projects/8f59f3c8410842e1b61a08a5d8b7c292 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:02.175566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:02.186420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4168e0-6b7d-4888-8f7a-0e46b77b3081 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:02.243217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:02.243841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:02.244122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:02.244392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] PATH_INFO: `/v3/users/9bd2cb266fba463a9157cc2cdcb98022` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:02.245520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd2cb266fba463a9157cc2cdcb98022)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:02.248067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:02.422424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd2cb266fba463a9157cc2cdcb98022'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.432363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd2cb266fba463a9157cc2cdcb98022'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.448004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd2cb266fba463a9157cc2cdcb98022'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.509979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd2cb266fba463a9157cc2cdcb98022'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.510611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e722ffcf-04df-4270-a77d-a4ed544ee29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd2cb266fba463a9157cc2cdcb98022'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:02.518486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 984/1962] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:02 2022] DELETE /identity/v3/users/9bd2cb266fba463a9157cc2cdcb98022 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:02.966802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:02.968255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:02.982938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e013d-85de-43c3-816c-17a66edfb223 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.078265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.078265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.078265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.078265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] PATH_INFO: `/v3/projects/694d1961124c42b9a410923c2f4bcf67` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.079403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694d1961124c42b9a410923c2f4bcf67)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.081593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.188926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694d1961124c42b9a410923c2f4bcf67', 'request_id': 'req-4b437c24-3729-4b54-85aa-ccffc0ad5377'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:03.194382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694d1961124c42b9a410923c2f4bcf67', 'request_id': 'req-4b437c24-3729-4b54-85aa-ccffc0ad5377'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:03.201518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694d1961124c42b9a410923c2f4bcf67', 'request_id': 'req-4b437c24-3729-4b54-85aa-ccffc0ad5377'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:03.276560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:03.281924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] The token cache is being invalidate because project 694d1961124c42b9a410923c2f4bcf67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:03.282217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:03.282939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4b437c24-3729-4b54-85aa-ccffc0ad5377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:03.284526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 979/1963] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:02 2022] DELETE /identity/v3/projects/694d1961124c42b9a410923c2f4bcf67 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:03.289427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.289890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.289981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.319594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:03.319594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:03.347344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:03.377200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 985/1964] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:03.384934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.385217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.385488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.405990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:03.407224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:03.415932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 980/1965] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:03.420222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.429891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da396-b469-49f0-985d-2f4c3d901844 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.434503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.435131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.435407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.435673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.436315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:03.436757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.438342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.451254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 986/1966] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:03.455970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.466913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc98e-d4eb-4ce4-a829-f97c62b21703 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.472545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.473222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.473500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.473764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.474388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.476432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.590758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 981/1967] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:03 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:03.595205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.605127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29b1e-6830-4535-a17b-ee22d9719c06 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.660668 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.661255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.661529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.661797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.662416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.664370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.720926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 987/1968] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:03 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:03.724931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.735956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6a0e-66a9-4ec9-b388-3f2f4c3b035d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.740585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.741170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.741444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.741710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.742408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.744026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.754605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 982/1969] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:03.758479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.769046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdd19-742a-45cf-8b75-1d4b130d2ada admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.773708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.774296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.774566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.774863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.775493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.777127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5d12b-3366-4536-b83b-498fe02e44ad admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.787624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 988/1970] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:03.791693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:03.801439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f4-f205-4169-beb0-f607ddbd44be admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:03.805948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:03.806583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.806914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.807255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] PATH_INFO: `/v3/projects/bf903b551b1e4fa1b5be2a7ff7db70e7/users/564805385138452c897fafd5fe976734/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.822315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf903b551b1e4fa1b5be2a7ff7db70e7, user_id=564805385138452c897fafd5fe976734, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:03.826054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:03.904119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 983/1971] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:03 2022] PUT /identity/v3/projects/bf903b551b1e4fa1b5be2a7ff7db70e7/users/564805385138452c897fafd5fe976734/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:03.909123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684825a-6f95-42b2-b146-42839a9c90da None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:03.909410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684825a-6f95-42b2-b146-42839a9c90da None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:03.909680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684825a-6f95-42b2-b146-42839a9c90da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:03.928545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9684825a-6f95-42b2-b146-42839a9c90da None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:03.929753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9684825a-6f95-42b2-b146-42839a9c90da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:03.990513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 989/1972] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:04.001258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:04.003265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:04.059724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0418e3-a5bb-4ddb-a00d-920f08e446e0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:04.063457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:04.064063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:04.064346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:04.064621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:04.073812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:04.076019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:04.085747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 984/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:05.586401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:05.586925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:05.587029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:05.616738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] MFA Rules not processed for user `564805385138452c897fafd5fe976734`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:05.617940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:05.656464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:05.699550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 990/1974] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:05.702763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:05.712360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e20c7-23c6-40d4-be67-874cd0703970 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:05.717635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:05.718215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:05.718490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:05.718794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:05.719474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:05.721354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:05.785524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 985/1975] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:05 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:05.789528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:05.800372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b3e3-fe7f-4d71-b4e9-6175e2440c4f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:05.857779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:05.858379 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:05.858653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:05.859217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:05.859932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:05.861865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:05.932254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 991/1976] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:05 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:05.936727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:05.947448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f8146-81b8-41f1-935a-2530a3c6b7a5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:05.952872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:05.961265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:05.961265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:05.961265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:05.961265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:05.961265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:05.967826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 986/1977] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:05.973097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:05.987011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13c9af-6ca0-44ba-aa15-e03f35c27099 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:05.991621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:05.992505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:05.992926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:05.993333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] PATH_INFO: `/v3/projects/4d788ddfb75f4d78863bd0df4e516890/users/8013e81ccd564efab246c5c7644ea9a3/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:06.013014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d788ddfb75f4d78863bd0df4e516890, user_id=8013e81ccd564efab246c5c7644ea9a3, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:06.015769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:06.095691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 992/1978] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:05 2022] PUT /identity/v3/projects/4d788ddfb75f4d78863bd0df4e516890/users/8013e81ccd564efab246c5c7644ea9a3/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:06.098349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:06.110387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ad32d-7912-4b4d-9de4-dc6e64fb08b9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:06.187393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:06.187941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:06.188219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:06.188491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:06.189158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:06.190755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:06.195978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:06.201369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 987/1979] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:06.210901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:06.219904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d0e8-f57d-421a-82eb-b05e6d4bf8a1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:06.224720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:06.225370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:06.225647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:06.225919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] PATH_INFO: `/v3/projects/4d788ddfb75f4d78863bd0df4e516890/users/8013e81ccd564efab246c5c7644ea9a3/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:06.227896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d788ddfb75f4d78863bd0df4e516890, user_id=8013e81ccd564efab246c5c7644ea9a3, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:06.231175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86a174-b8da-46cc-982f-f7bf9ae269a5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:06.241790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 993/1980] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:06 2022] PUT /identity/v3/projects/4d788ddfb75f4d78863bd0df4e516890/users/8013e81ccd564efab246c5c7644ea9a3/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:06.755358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:06.759990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:06.772738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70d818-6699-445c-8340-2a13ba4a954c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:06.842949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:06.843912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:06.844212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:06.844473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] PATH_INFO: `/v3/users/c34d7cb26c7648ea9571256bb604c3f1` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:06.851979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34d7cb26c7648ea9571256bb604c3f1)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:06.851979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:07.092428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34d7cb26c7648ea9571256bb604c3f1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:07.111248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34d7cb26c7648ea9571256bb604c3f1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:07.111909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34d7cb26c7648ea9571256bb604c3f1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:07.118868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34d7cb26c7648ea9571256bb604c3f1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:07.119508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34d7cb26c7648ea9571256bb604c3f1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:07.218283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 988/1981] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:06 2022] DELETE /identity/v3/users/c34d7cb26c7648ea9571256bb604c3f1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:07.219258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a378f-7015-4bc3-8fee-4ba9156d0e3b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:07.219739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a378f-7015-4bc3-8fee-4ba9156d0e3b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:07.219924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a378f-7015-4bc3-8fee-4ba9156d0e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:07.247759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a48a378f-7015-4bc3-8fee-4ba9156d0e3b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:07.249437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a48a378f-7015-4bc3-8fee-4ba9156d0e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.336966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 994/1982] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:07.344284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:07.349298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.439920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0533963-c399-4576-9c4a-76ef2cdd2b6b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:07.444171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:07.444535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:07.444815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:07.445365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:07.458079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:07.459703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:07.472149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 989/1983] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:07.649553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:07.650229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:07.651335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:07.726367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] MFA Rules not processed for user `8013e81ccd564efab246c5c7644ea9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:07.727965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.740905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:07.823256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 995/1984] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:07.915677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:07.918221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.948758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:07.957025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:07.959703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.982204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c7cf0-1707-4ace-8db3-95859c6704d4 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:07.988441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:07.988441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:07.988441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:07.988441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:07.989001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b735223-4392-4f6a-a071-38abd73b5dbb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:07.991127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:07.997678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:07.998688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:07.999098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:07.999472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] PATH_INFO: `/v3/projects/07ce79e5bbe04114907a30ebda502b83` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:08.000868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ce79e5bbe04114907a30ebda502b83)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:08.003115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:08.062836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:08.067700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:08.073576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 990/1985] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:08.122254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ce79e5bbe04114907a30ebda502b83', 'request_id': 'req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.128136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ce79e5bbe04114907a30ebda502b83', 'request_id': 'req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.134263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ce79e5bbe04114907a30ebda502b83', 'request_id': 'req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.208926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] The token cache is being invalidate because project 07ce79e5bbe04114907a30ebda502b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:08.210362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.211893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.216202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 996/1986] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:07 2022] DELETE /identity/v3/projects/07ce79e5bbe04114907a30ebda502b83 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:08.220260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:08.232331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e5d5-039b-4b37-b80d-0f282e698364 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:08.291514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:08.292146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:08.292420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:08.293489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] PATH_INFO: `/v3/users/e79885951bf0441d928da45986572c23` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:08.294979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79885951bf0441d928da45986572c23)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:08.298441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:08.540311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79885951bf0441d928da45986572c23'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.554440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79885951bf0441d928da45986572c23'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.555197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79885951bf0441d928da45986572c23'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.566951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79885951bf0441d928da45986572c23'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.568329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-1cf49b3d-e6a6-4eda-bcd9-3b022db3f3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79885951bf0441d928da45986572c23'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:08.679452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 991/1987] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:08 2022] DELETE /identity/v3/users/e79885951bf0441d928da45986572c23 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:09.115823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.119403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:09.136641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea9d4-dd5b-4684-9aa4-36e202753ff6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.196330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.196961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.197243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.197512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] PATH_INFO: `/v3/projects/04d559cdab3c430390cee060f20ee42d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.198648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d559cdab3c430390cee060f20ee42d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.201355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.252375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:09.315961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:09.328213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d559cdab3c430390cee060f20ee42d', 'request_id': 'req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:09.333776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d559cdab3c430390cee060f20ee42d', 'request_id': 'req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:09.341032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d559cdab3c430390cee060f20ee42d', 'request_id': 'req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:09.429072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] The token cache is being invalidate because project 04d559cdab3c430390cee060f20ee42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:09.429356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:09.429997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dfeaf3df-a7aa-4885-9bd9-07d03ad6ff17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:09.432590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 997/1988] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:09 2022] DELETE /identity/v3/projects/04d559cdab3c430390cee060f20ee42d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:09.436446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.436735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.437016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.457254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:09.461458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:09.519579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 992/1989] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:09.529545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.529821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.530087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.550014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:09.551217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:09.559825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 998/1990] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:09.564169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.575452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17d06f-b5da-45d5-afa9-f684c1ad136e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.580097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.581788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.581788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.581788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.582036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:09.582196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.584353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.596065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 993/1991] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:09.600643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.609914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aeb52-6afc-4d13-bda7-10a281fea9a2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.614651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.615266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.615542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.615810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.616495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.618370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.709181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 999/1992] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:09 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:09.710590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.721067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38d30-cfe7-4676-ace9-9e8a9c8c32a5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.779079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.779718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.779994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.780266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.780909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.782904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.881530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 994/1993] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:09 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:09.885697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.897835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6bf99-6a9c-4b5a-9e1e-c7d41c57b5a0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.902838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.903520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.903865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.904201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.904927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.906652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.930150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1000/1994] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:09.934867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.947660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bdfc0-dc24-4ec3-8240-1829568e8191 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.952639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.953231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.953505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.953772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:09.954401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:09.956003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7e649-6b7a-4238-ae92-7679038adc91 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:09.966686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 995/1995] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:09.970861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:09.981639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd543-e9c9-4762-8e12-211cd55931eb admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:09.987317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:09.987989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:09.988270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:09.988543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] PATH_INFO: `/v3/projects/41095bbec10d4d0fbd4f95bca56a87cb/users/391e2e5c1f594ff5a47b18913b0c5383/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.004405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41095bbec10d4d0fbd4f95bca56a87cb, user_id=391e2e5c1f594ff5a47b18913b0c5383, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.007097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.080703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1001/1996] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:09 2022] PUT /identity/v3/projects/41095bbec10d4d0fbd4f95bca56a87cb/users/391e2e5c1f594ff5a47b18913b0c5383/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:10.091468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.096858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.096858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.119205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] MFA Rules not processed for user `391e2e5c1f594ff5a47b18913b0c5383`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:10.120386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:10.165947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:10.183023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 996/1997] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:10.188235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.249170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479c1c-27e3-4e40-b11b-18e462053d59 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.253848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.254446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.254746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.255023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.255647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.257555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.337080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1002/1998] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:10 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:10.346467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.349582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520219b-4f13-4799-94a3-de43be1e6a4b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.408830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.409433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.409709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.409976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.410596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.412552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.509688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 997/1999] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:10 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:10.515956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.526328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c010ea-0b9d-4cc8-b62d-7c390776e49f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.531923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.532514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.532809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.533076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.533707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.535328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.546851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1003/2000] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:10.551101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.561825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba29abc-74ca-4f32-92ed-a15f853a8123 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.566495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.567171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.567441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.567710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] PATH_INFO: `/v3/projects/9608b69f09d54cf683f8b3321d7731db/users/aa5c42dd321f437a9b3ce3d82e0528c8/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.582424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9608b69f09d54cf683f8b3321d7731db, user_id=aa5c42dd321f437a9b3ce3d82e0528c8, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.585118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.657964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 998/2001] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:10 2022] PUT /identity/v3/projects/9608b69f09d54cf683f8b3321d7731db/users/aa5c42dd321f437a9b3ce3d82e0528c8/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:10.661927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.672994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a047a4-0581-4500-a520-2c644dec577f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.701527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:10.728307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.728942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.729232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.729501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.730177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.732146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ac85-bc68-4708-868e-076d97c83686 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.742556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1004/2002] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:10.746698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.756835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19a1be-d574-4357-b186-8f1af631086f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.762092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.762757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.763038 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.763311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] PATH_INFO: `/v3/projects/9608b69f09d54cf683f8b3321d7731db/users/aa5c42dd321f437a9b3ce3d82e0528c8/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.765221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9608b69f09d54cf683f8b3321d7731db, user_id=aa5c42dd321f437a9b3ce3d82e0528c8, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.767885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:10.778541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 999/2003] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:10 2022] PUT /identity/v3/projects/9608b69f09d54cf683f8b3321d7731db/users/aa5c42dd321f437a9b3ce3d82e0528c8/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:10.789694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.789997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.790255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.820085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] MFA Rules not processed for user `aa5c42dd321f437a9b3ce3d82e0528c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:10.821165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:10.891739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1005/2004] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:10.903651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:10.905515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:10.976765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b35dd-078f-40e8-845e-34c1d06d9980 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:10.981140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:10.981896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:10.982269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:10.982629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:10.992252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:10.993735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46818f97-6329-43f4-ab25-de7451ad960d service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:11.003959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1000/2005] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:11.068768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:11.132551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d3674-fca4-456e-9311-cc1ab5cd946a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:11.137889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:11.138502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:11.138810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:11.139075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] PATH_INFO: `/v3/users/391e2e5c1f594ff5a47b18913b0c5383` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:11.140212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] RBAC: Authorizing `identity:delete_user(user_id=391e2e5c1f594ff5a47b18913b0c5383)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:11.142109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:11.324618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391e2e5c1f594ff5a47b18913b0c5383'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:11.329879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:11.331741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391e2e5c1f594ff5a47b18913b0c5383'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:11.343162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391e2e5c1f594ff5a47b18913b0c5383'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:11.426844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391e2e5c1f594ff5a47b18913b0c5383'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:11.427495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391e2e5c1f594ff5a47b18913b0c5383'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:11.434421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1006/2006] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:11 2022] DELETE /identity/v3/users/391e2e5c1f594ff5a47b18913b0c5383 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:11.438938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:11.439465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:11.439465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:11.457760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:11.458971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:11.507335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:11.524839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1001/2007] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:11.534629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:11.536088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:11.595428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc824d5-3a98-4be6-9347-e1b64fc209af admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:11.599050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:11.599652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:11.599931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:11.600194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:11.609912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:11.611381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:11.621624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1007/2008] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:12.027928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:12.029593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:12.041383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aec8e-11d2-4121-a625-b396c71e5951 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:12.047605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:12.048358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:12.048639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:12.048911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] PATH_INFO: `/v3/projects/41095bbec10d4d0fbd4f95bca56a87cb` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:12.050010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41095bbec10d4d0fbd4f95bca56a87cb)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:12.051981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:12.156835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41095bbec10d4d0fbd4f95bca56a87cb', 'request_id': 'req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.162519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41095bbec10d4d0fbd4f95bca56a87cb', 'request_id': 'req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.169416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41095bbec10d4d0fbd4f95bca56a87cb', 'request_id': 'req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.223042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] The token cache is being invalidate because project 41095bbec10d4d0fbd4f95bca56a87cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:12.223361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.224079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.225528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1002/2009] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:12 2022] DELETE /identity/v3/projects/41095bbec10d4d0fbd4f95bca56a87cb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:12.229502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:12.239693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ee73a-c25d-477e-a411-5c53345e5c23 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:12.295719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:12.296335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:12.296614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:12.296886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] PATH_INFO: `/v3/users/aa5c42dd321f437a9b3ce3d82e0528c8` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:12.297994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5c42dd321f437a9b3ce3d82e0528c8)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:12.299944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:12.430992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5c42dd321f437a9b3ce3d82e0528c8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.438614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5c42dd321f437a9b3ce3d82e0528c8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.451255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5c42dd321f437a9b3ce3d82e0528c8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.460249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:12.492206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5c42dd321f437a9b3ce3d82e0528c8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.492830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-c48e5ca0-2586-4463-a386-f6c7f529a81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5c42dd321f437a9b3ce3d82e0528c8'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:12.499368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1008/2010] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:12 2022] DELETE /identity/v3/users/aa5c42dd321f437a9b3ce3d82e0528c8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:12.905403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:12.906861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:12.920812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17c49-09fd-4d94-8bb8-9fc5420b496b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:12.978668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:12.979235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:12.979510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:12.980709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] PATH_INFO: `/v3/projects/9608b69f09d54cf683f8b3321d7731db` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:12.982203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9608b69f09d54cf683f8b3321d7731db)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:12.984415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.102637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9608b69f09d54cf683f8b3321d7731db', 'request_id': 'req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:13.109387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9608b69f09d54cf683f8b3321d7731db', 'request_id': 'req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:13.115586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9608b69f09d54cf683f8b3321d7731db', 'request_id': 'req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:13.185482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:13.190819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] The token cache is being invalidate because project 9608b69f09d54cf683f8b3321d7731db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:13.190819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:13.191436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3fa8b3cc-d9a7-4bf4-b6d4-8524358fea21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:13.193037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1003/2011] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:12 2022] DELETE /identity/v3/projects/9608b69f09d54cf683f8b3321d7731db => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:13.197846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.198136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.198418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.223779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:13.225002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:13.286835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1009/2012] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:13.295831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.296198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.296657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.318566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:13.319799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:13.330614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1004/2013] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:13.338679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.349533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c72ce-cc52-4175-a492-396ef61d3a3c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.354068 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.354650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.354961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.356422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.357063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:13.357360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.359298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.370600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1010/2014] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:13.375236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.384722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383cd4c-35ad-4987-b2d6-ef5ef1c95359 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.389976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.393091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.393091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.393091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.393091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.393697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.483846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1005/2015] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:13.488037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.498913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9a28-6a4d-4c7e-bbb1-e7554761020b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.556928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.557530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.557806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.558075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.558984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.560958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.647018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1011/2016] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:13.651122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.662137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7deeda-b4e6-4d83-b6f6-613f846fc02b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.666925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.667644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.668474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.668717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.669355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.670958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.682617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1006/2017] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:13.686831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.696976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70175d-e208-4967-a4c1-dac399fd732c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.702527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.703137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.703411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.703675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.704302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.706018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af320156-65e2-43ce-9748-e4f9ecfe88ed admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.717292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1012/2018] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:13.721525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:13.731720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2930-fe3f-4d80-aa79-eb343236139a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:13.736606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:13.737246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.737527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.737802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] PATH_INFO: `/v3/projects/30a7e638924b426090d827726332fdcd/users/034502607d8845b79010b6406f93529b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.752250 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a7e638924b426090d827726332fdcd, user_id=034502607d8845b79010b6406f93529b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:13.755057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:13.852488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1007/2019] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:13 2022] PUT /identity/v3/projects/30a7e638924b426090d827726332fdcd/users/034502607d8845b79010b6406f93529b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:13.863060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:13.863341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:13.863628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:13.891365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] MFA Rules not processed for user `034502607d8845b79010b6406f93529b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:13.892558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:13.938788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:13.957235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1013/2020] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:13.961978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.027265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517e45-073e-4ce6-b111-7dca085ce4c6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.032559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.033156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.033432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.033702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.034423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.036361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.126451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1008/2021] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:13 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:14.127606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.139585 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46a5a1-568e-4579-ad0f-88b5f0f58630 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.195752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.196350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.196633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.196888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.197506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.200525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.310391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1014/2022] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:14 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:14.314460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.325641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18535c6-51b6-4b20-ac73-2dfd62bbd4cf admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.330278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.330919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.331197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.331496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.332130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.333708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.344499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1009/2023] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:14.348542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.358485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac408a-4d24-4f01-b6ed-8c3e5d14554b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.363796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.364494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.364774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.365042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] PATH_INFO: `/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2/users/fb5e0da4b4014fda8a8186eeb7d52a6f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.380851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd73c04f2814c6a9c7cd2b4a93088e2, user_id=fb5e0da4b4014fda8a8186eeb7d52a6f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.385012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.436792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1015/2024] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:14 2022] PUT /identity/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2/users/fb5e0da4b4014fda8a8186eeb7d52a6f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:14.440373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.450852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b208ca-eb31-4e0f-9d0d-cc61c9baf0c4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.516209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.516820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.517097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.517367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.520825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.520825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237fdb5-b2c1-49ae-9b8e-2e0bf9ef93b9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.529687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1010/2025] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:14.533876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.544672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41907c0b-bf4b-48ab-8964-bd52ba8a352e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.549522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.550178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.552181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.552181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] PATH_INFO: `/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2/users/fb5e0da4b4014fda8a8186eeb7d52a6f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.552641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd73c04f2814c6a9c7cd2b4a93088e2, user_id=fb5e0da4b4014fda8a8186eeb7d52a6f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.555335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.566911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1016/2026] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:14 2022] PUT /identity/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2/users/fb5e0da4b4014fda8a8186eeb7d52a6f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:14.578170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.578465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.578764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.592671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:14.607334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] MFA Rules not processed for user `fb5e0da4b4014fda8a8186eeb7d52a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:14.608554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:14.679600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1011/2027] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:14.697364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.699147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:14.758389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13135-67e3-43b8-b6e7-1efb5c282116 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.762457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.763092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.763365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.763637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.772424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.773846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.783372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1017/2028] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:14.884244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:14.893600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d461c-85ac-4401-b005-35eded4e0683 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:14.897104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:14.897697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:14.897972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:14.898241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:14.955986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:14.957429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:14.967310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1012/2029] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:17.307609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:17.310348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:17.322006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a32d3-aa6c-4d8e-811c-b8554f6cac37 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:17.328096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:17.328701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:17.329592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:17.329592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:17.331128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:17.375034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:17.395652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:17.397116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f0daa-359b-4e4e-909d-167bc3a86167 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:17.413504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1018/2030] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:21.272107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:21.273495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:21.341542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1881e-ec76-4091-afb2-23ed94dbf0ae service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:21.346325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:21.347083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:21.347359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:21.347630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] PATH_INFO: `/v3/users/6ff88a42c4f24675b77406235e61c294` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:21.348754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff88a42c4f24675b77406235e61c294)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:21.350674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:21.540374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:21.555242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff88a42c4f24675b77406235e61c294'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:21.568058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff88a42c4f24675b77406235e61c294'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:21.569450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff88a42c4f24675b77406235e61c294'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:21.576254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff88a42c4f24675b77406235e61c294'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:21.577305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff88a42c4f24675b77406235e61c294'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:21.685637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1013/2031] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:21 2022] DELETE /identity/v3/users/6ff88a42c4f24675b77406235e61c294 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:21.690445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:21.690954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:21.691208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:21.711638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:21.712848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:21.781512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1019/2032] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:21.791382 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:21.792878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:21.854896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950b0a2-8162-438c-88b0-4c344f6cd6b7 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:21.859569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:21.860165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:21.860437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:21.860701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:21.870542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:21.872004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:21.881781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1014/2033] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:22.093806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:22.096991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:22.123603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251432d-c106-435f-8a47-12b86d8e6af6 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:22.134248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:22.139689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:22.139689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:22.139689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] PATH_INFO: `/v3/users/034502607d8845b79010b6406f93529b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:22.140642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=034502607d8845b79010b6406f93529b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:22.145290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:22.322792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:22.324193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:22.354856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0faf6-0802-4d32-80e3-44eafb99ff92 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:22.360536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:22.361167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:22.361520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:22.361703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] PATH_INFO: `/v3/projects/8e07f9d1cb104e298395fd112326f00f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:22.362849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e07f9d1cb104e298395fd112326f00f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:22.364799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:22.407100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034502607d8845b79010b6406f93529b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.414459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034502607d8845b79010b6406f93529b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.427010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034502607d8845b79010b6406f93529b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.483479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e07f9d1cb104e298395fd112326f00f', 'request_id': 'req-4cdb029a-a3a6-4763-b662-1c0ae48fa869'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.490954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e07f9d1cb104e298395fd112326f00f', 'request_id': 'req-4cdb029a-a3a6-4763-b662-1c0ae48fa869'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.499460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e07f9d1cb104e298395fd112326f00f', 'request_id': 'req-4cdb029a-a3a6-4763-b662-1c0ae48fa869'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.543986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034502607d8845b79010b6406f93529b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.544600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-7781687f-56a7-4e92-84ee-8cf1c32b92b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034502607d8845b79010b6406f93529b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.551032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1020/2034] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:22 2022] DELETE /identity/v3/users/034502607d8845b79010b6406f93529b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:22.555659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2865-022c-482c-9be6-65224c1f42db None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:22.556072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2865-022c-482c-9be6-65224c1f42db None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:22.556419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2865-022c-482c-9be6-65224c1f42db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:22.575175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-568e2865-022c-482c-9be6-65224c1f42db None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:22.576353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-568e2865-022c-482c-9be6-65224c1f42db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:22.628367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:22.634423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] The token cache is being invalidate because project 8e07f9d1cb104e298395fd112326f00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:22.634423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.634972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.636628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1015/2035] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:22 2022] DELETE /identity/v3/projects/8e07f9d1cb104e298395fd112326f00f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:22.641315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:22.642804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:22.648366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1021/2036] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:22.659512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e2865-022c-482c-9be6-65224c1f42db None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:22.663251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-568e2865-022c-482c-9be6-65224c1f42db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:22.693940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:22.722696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb029a-a3a6-4763-b662-1c0ae48fa869 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:22.739203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:22.740053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:22.740466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:22.740830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] PATH_INFO: `/v3/users/5af99d9358ad4529868cdfdc6dbc4ba2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:22.742052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e2865-022c-482c-9be6-65224c1f42db None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:22.742531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af99d9358ad4529868cdfdc6dbc4ba2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:22.744956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:22.746444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:22.747887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:22.748121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:22.749013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:22.760086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:22.761738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:22.785556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1022/2037] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:17:22.964253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af99d9358ad4529868cdfdc6dbc4ba2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.984642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af99d9358ad4529868cdfdc6dbc4ba2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:22.984642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af99d9358ad4529868cdfdc6dbc4ba2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.001115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af99d9358ad4529868cdfdc6dbc4ba2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.003326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af99d9358ad4529868cdfdc6dbc4ba2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.107112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1016/2038] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:22 2022] DELETE /identity/v3/users/5af99d9358ad4529868cdfdc6dbc4ba2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:23.221032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:23.226854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:23.243883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb0fa-5a70-474d-9ba6-898c4ef91f88 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:23.313061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:23.313696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:23.313971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:23.314246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] PATH_INFO: `/v3/projects/30a7e638924b426090d827726332fdcd` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:23.315391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a7e638924b426090d827726332fdcd)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:23.317917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:23.494487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a7e638924b426090d827726332fdcd', 'request_id': 'req-04199a22-3b38-464a-b7ac-ed1453c149ad'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.505232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a7e638924b426090d827726332fdcd', 'request_id': 'req-04199a22-3b38-464a-b7ac-ed1453c149ad'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.516031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a7e638924b426090d827726332fdcd', 'request_id': 'req-04199a22-3b38-464a-b7ac-ed1453c149ad'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.605652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] The token cache is being invalidate because project 30a7e638924b426090d827726332fdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:23.605997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.606618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.608784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1023/2039] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:23 2022] DELETE /identity/v3/projects/30a7e638924b426090d827726332fdcd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:23.613062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:23.614493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:23.628162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be4470-ba77-4d5e-bea3-474d72de791e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:23.700540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:23.702253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:23.704247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:23.705392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:23.705591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:23.705888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] PATH_INFO: `/v3/users/fb5e0da4b4014fda8a8186eeb7d52a6f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:23.708165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5e0da4b4014fda8a8186eeb7d52a6f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:23.711471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:23.747602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199a22-3b38-464a-b7ac-ed1453c149ad admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:23.753380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:23.754964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:23.755305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:23.755750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] PATH_INFO: `/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:23.757138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a968bfc7e642f4bf83f6c5e87ce04d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:23.759684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:23.785856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:23.939428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5e0da4b4014fda8a8186eeb7d52a6f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.946218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a968bfc7e642f4bf83f6c5e87ce04d', 'request_id': 'req-4dfb865e-3a02-4ce2-9762-2186cd68f270'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.950219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:23.953583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a968bfc7e642f4bf83f6c5e87ce04d', 'request_id': 'req-4dfb865e-3a02-4ce2-9762-2186cd68f270'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.959492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5e0da4b4014fda8a8186eeb7d52a6f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.960531 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5e0da4b4014fda8a8186eeb7d52a6f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.971429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5e0da4b4014fda8a8186eeb7d52a6f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.972814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9b43d307-38d6-4196-a5a7-3c51daaa785c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5e0da4b4014fda8a8186eeb7d52a6f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:23.973060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a968bfc7e642f4bf83f6c5e87ce04d', 'request_id': 'req-4dfb865e-3a02-4ce2-9762-2186cd68f270'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.059236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1017/2040] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:23 2022] DELETE /identity/v3/users/fb5e0da4b4014fda8a8186eeb7d52a6f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:24.166998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] The token cache is being invalidate because project 34a968bfc7e642f4bf83f6c5e87ce04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:24.167562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.168349 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4dfb865e-3a02-4ce2-9762-2186cd68f270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.173815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1024/2041] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:23 2022] DELETE /identity/v3/projects/34a968bfc7e642f4bf83f6c5e87ce04d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:24.177231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.177231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.177231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.203798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:24.205047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:24.295638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1018/2042] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:24.303536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.303928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.304295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.329507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:24.331080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:24.340817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1025/2043] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:24.348562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.374540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ead6f-a61b-4717-b623-9f29635593cb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.383098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.383793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.384137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.384474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.385237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:24.385753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.387462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.403316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1019/2044] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:24.404618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.414275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35036f-113d-43c7-b6d5-b6be40ac38b5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.419859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.420713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.421337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.421610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.422247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.431461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.564460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.564984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1026/2045] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:24.567448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:24.575203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.596135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feea13-a817-4dc5-8fec-3d5397eadcb2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.661951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a0f43-8626-4766-b08e-1e53e2c1de1b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.667718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.669927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.669927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.669927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.669927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.672152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.679306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.680708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.681258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.681763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] PATH_INFO: `/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.683367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd73c04f2814c6a9c7cd2b4a93088e2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.685747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.747563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1027/2046] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:24.749659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.761099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210ef4c-5a43-464c-aaa4-505c46412427 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.767097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.767815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.768185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.768559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.769262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.770984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.782637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1028/2047] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:24.786304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.797854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed5b9-84ee-4030-bbdf-35ae99acb7b1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.810404 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd73c04f2814c6a9c7cd2b4a93088e2', 'request_id': 'req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.817495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd73c04f2814c6a9c7cd2b4a93088e2', 'request_id': 'req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.834549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd73c04f2814c6a9c7cd2b4a93088e2', 'request_id': 'req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.880658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:24.887063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.887669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.887952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.888222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.888863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.890890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.905188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1029/2048] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:24.906411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:24.916419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19759e-a9a2-4032-902d-dbcd25d67d96 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:24.922831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:24.923476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.923748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.924014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] PATH_INFO: `/v3/projects/2e00e84dfa0347ff830c294711d68ff4/users/a003ad777048416d8515c84367a429a6/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.936853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] The token cache is being invalidate because project 1dd73c04f2814c6a9c7cd2b4a93088e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:24.937552 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.938872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98a05509-e9d8-4bee-9f71-c5c3ab1d4bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:24.941222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1020/2049] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:24 2022] DELETE /identity/v3/projects/1dd73c04f2814c6a9c7cd2b4a93088e2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:24.946243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685132e9-eb79-443c-a069-aae386dac412 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.946578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685132e9-eb79-443c-a069-aae386dac412 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.947266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685132e9-eb79-443c-a069-aae386dac412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:24.955496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e00e84dfa0347ff830c294711d68ff4, user_id=a003ad777048416d8515c84367a429a6, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:24.958176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:24.974407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-685132e9-eb79-443c-a069-aae386dac412 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:24.975849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-685132e9-eb79-443c-a069-aae386dac412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:24.985631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1021/2050] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:24.993682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:24.994020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:24.994613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.016988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:25.018550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:25.030913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1022/2051] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:25.033917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.043236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4ffc2-acf9-4a8a-989e-8f571cd168e2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.048691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.049286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.049560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.049834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.050460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:25.050779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.052368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e911-8089-431a-ba81-3230252e2084 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.070033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1030/2052] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:24 2022] PUT /identity/v3/projects/2e00e84dfa0347ff830c294711d68ff4/users/a003ad777048416d8515c84367a429a6/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:25.072886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1023/2053] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:25.081758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.081897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.082227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.082560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.146728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] MFA Rules not processed for user `a003ad777048416d8515c84367a429a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:25.147748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:25.150749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:25.176031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a39-ca38-4e0e-9c30-76d3c19ee7d8 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.181389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.182126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.182513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.183009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.183731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.186429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.252050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1024/2054] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:25.256549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.257935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:25.274907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18352cd6-d64d-4ccc-854f-4f5428dcaa5e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.284082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.284800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.285160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.285881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.286213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.288285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.338156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1031/2055] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/projects => generated 377 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:25.342725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.407128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fb231-9187-4291-bb63-b35b3b74667c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.412425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.413046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.413323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.413607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.414219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.416201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.417042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1025/2056] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:25.421873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.500826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37a6c6-bf16-4c40-b7db-71eb9a09b6be admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.505778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.506393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.506662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.506954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.508351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.510305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.522636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1032/2057] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:25.526513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.537569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25315ca-fa41-4c9e-a7ed-56cf079774de admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.548413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.548413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.548413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.548413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.548915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.548915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.558766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1033/2058] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:25.563159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.577953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b269b-a8b2-4271-a263-fcd209457bfb admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.594360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.594360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.594360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.594360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.594880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.595773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.605888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1034/2059] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:25.614564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.626099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1026/2060] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:25.627075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.631419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721712-e66e-4a5e-bb38-b2fc76e13e7d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.638665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.639858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.640135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.640413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] PATH_INFO: `/v3/projects/31ac2cfe44ce4948b3fdc74ba913b8bf/users/e1d03287e26e4e3dbff823530de33908/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.642101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac24b-0548-40ac-a468-e84f4383dc4d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.652446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.654298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.654650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.655515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.656172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.661306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.661476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ac2cfe44ce4948b3fdc74ba913b8bf, user_id=e1d03287e26e4e3dbff823530de33908, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.663599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1096343-dee0-49c7-907b-df7ce935c570 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.672658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1027/2061] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:25.676849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.691949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2e437-3b56-48c2-bdaf-b83454587f77 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.692557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.692722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.693001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.693317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] PATH_INFO: `/v3/projects/5451430eca6c49f38d94d43138acab87/users/d3ffc23790754f1c942a225ac70b95ff/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.709824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5451430eca6c49f38d94d43138acab87, user_id=d3ffc23790754f1c942a225ac70b95ff, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.712523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.757594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1035/2062] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:25 2022] PUT /identity/v3/projects/31ac2cfe44ce4948b3fdc74ba913b8bf/users/e1d03287e26e4e3dbff823530de33908/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:25.760743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544408e1-e85b-4789-9590-0918314d4149 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.761085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544408e1-e85b-4789-9590-0918314d4149 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.761394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544408e1-e85b-4789-9590-0918314d4149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.794845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1028/2063] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:25 2022] PUT /identity/v3/projects/5451430eca6c49f38d94d43138acab87/users/d3ffc23790754f1c942a225ac70b95ff/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:25.795266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-544408e1-e85b-4789-9590-0918314d4149 None None] MFA Rules not processed for user `e1d03287e26e4e3dbff823530de33908`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:25.800985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-544408e1-e85b-4789-9590-0918314d4149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:25.801097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.871334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f81ff9-380b-4d41-a4fa-266489729c06 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.877867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.878482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.878798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.879072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.880238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.881976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.888606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1036/2064] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:25.895719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1029/2065] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:25.899888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544408e1-e85b-4789-9590-0918314d4149 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.901137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:25.906365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:25.918294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544408e1-e85b-4789-9590-0918314d4149 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.924364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:25.925037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.925316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.925892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] PATH_INFO: `/v3/projects/5451430eca6c49f38d94d43138acab87/users/d3ffc23790754f1c942a225ac70b95ff/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.927903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5451430eca6c49f38d94d43138acab87, user_id=d3ffc23790754f1c942a225ac70b95ff, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:25.930980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0170f-461c-40bd-805c-a85a9fcc0cc9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:25.944351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1037/2066] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:25 2022] PUT /identity/v3/projects/5451430eca6c49f38d94d43138acab87/users/d3ffc23790754f1c942a225ac70b95ff/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:25.956353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:25.956647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:25.956926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:25.988187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204f6d-5c43-4d03-9a9f-13df686272b0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:25.991322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] MFA Rules not processed for user `d3ffc23790754f1c942a225ac70b95ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:25.992748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:26.071074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:26.077170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:26.077170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:26.077170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:26.086285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1038/2067] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:26.095218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:26.097257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:26.146728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:26.148607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09734f-b496-4bf1-84d9-f34751879d4a service glance] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:26.159925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1030/2068] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:26.171990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc885be2-0e39-45b6-9127-dea04a05660b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:26.176295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:26.176978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:26.177263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:26.177568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:26.185212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:26.193155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:26.194812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19232e-d61a-442e-9fa5-b48b53113a6e service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:26.207492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1039/2069] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:26.355062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:27.309262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:28.379582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:28.380216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:28.380216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:28.381778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1031/2070] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:17:28.394116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:28.394748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:28.394748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:28.406578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:28.421732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:28.423211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:28.498410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1040/2071] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:17:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:28.509983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:28.511960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:28.535475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:28.539556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:28.544095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395dee5-98b6-4bab-8395-b72e92872527 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:28.550098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:28.550956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:28.556178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:28.556178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:28.562467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:28.564086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:28.574657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1032/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:28.630603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf34539-e784-472f-a3ad-25a3536239f8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:28.639637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:28.639637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:28.639637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:28.639637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] PATH_INFO: `/v3/users/5f405a8b7cd64c7e9615a528a0d2484b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:28.640116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f405a8b7cd64c7e9615a528a0d2484b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:28.640515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:28.827567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f405a8b7cd64c7e9615a528a0d2484b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:28.834463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f405a8b7cd64c7e9615a528a0d2484b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:28.851442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f405a8b7cd64c7e9615a528a0d2484b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:28.957701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f405a8b7cd64c7e9615a528a0d2484b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:28.959187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9b1a2359-e9c9-4a5f-82b3-aaa0dc31e10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f405a8b7cd64c7e9615a528a0d2484b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:28.970834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1041/2073] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:28 2022] DELETE /identity/v3/users/5f405a8b7cd64c7e9615a528a0d2484b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:29.272703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:29.285241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da445-d939-4bbf-bbb7-6923fff96f62 service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:29.377429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:29.378076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.378353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.378743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] PATH_INFO: `/v3/projects/ad80b76c328d44aa9479a98367f0f65c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.379897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad80b76c328d44aa9479a98367f0f65c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:29.381823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:29.480490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad80b76c328d44aa9479a98367f0f65c', 'request_id': 'req-22e5d374-01e6-4dd7-90d8-c08d3ee48020'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.488597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad80b76c328d44aa9479a98367f0f65c', 'request_id': 'req-22e5d374-01e6-4dd7-90d8-c08d3ee48020'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.497102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad80b76c328d44aa9479a98367f0f65c', 'request_id': 'req-22e5d374-01e6-4dd7-90d8-c08d3ee48020'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.567272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] The token cache is being invalidate because project ad80b76c328d44aa9479a98367f0f65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:29.567607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.568412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-22e5d374-01e6-4dd7-90d8-c08d3ee48020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.570016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1033/2074] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:29 2022] DELETE /identity/v3/projects/ad80b76c328d44aa9479a98367f0f65c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:29.580448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.580448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.580448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.595825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:29.597039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:29.609645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:29.655773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1042/2075] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:29.663446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.663726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.663991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.685659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:29.687590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:29.696011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1034/2076] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:29.700423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:29.704911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:29.706262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:29.711034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27d60-2972-47f6-b9ec-1ebb2670f4b1 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:29.715981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:29.716787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.717095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.717609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.718208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aee40b-ab41-4fed-a472-21026db1c936 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:29.718393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:29.718689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:29.720452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:29.726324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:29.727048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.727328 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.727602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] PATH_INFO: `/v3/users/a003ad777048416d8515c84367a429a6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.728745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a003ad777048416d8515c84367a429a6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:29.730723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:29.733276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1043/2077] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:29.738911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:29.754009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b49307-eb6f-44b0-905a-c44367f58376 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:29.759488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:29.760363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.760657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.761009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.761543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:29.763468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:29.854917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1044/2078] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:29 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:29.860829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:29.901689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a003ad777048416d8515c84367a429a6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.911473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:29.916091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a003ad777048416d8515c84367a429a6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.916928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a003ad777048416d8515c84367a429a6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.925504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a003ad777048416d8515c84367a429a6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.925918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebe236-c7ac-44ed-9097-57a19a8b44f7 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:29.926361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0c13850d-131f-48c7-872a-50bcda88477a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a003ad777048416d8515c84367a429a6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:29.931054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:29.931655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.931952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.932216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:29.932907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:29.934883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:29.973612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1035/2079] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:29 2022] DELETE /identity/v3/users/a003ad777048416d8515c84367a429a6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:29.980400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:29.980682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:29.985232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.008526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1045/2080] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:29 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:30.010804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:30.011986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.012870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.014599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.027360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e687f-7b87-4da2-979e-f883360e3bfd admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.079874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1036/2081] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:30.095044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.096488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.105421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.106039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.106312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.106580 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.107260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.109920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.120608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1046/2082] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:30.124810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.134764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaa1b3-ba1d-481c-b1fa-a7e3b3c5b1fd admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.143791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.144403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.145076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.145076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.145582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.147257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.150163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:30.162489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1047/2083] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:30.166930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.174654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8cd0-4639-496d-83af-b1236d813e01 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.177298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d6c18-fef9-4bee-81d6-eb242c45400d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.179461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.180190 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.180408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.180733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.182855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.184642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.185023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.185391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] PATH_INFO: `/v3/projects/b23fb97c7c4049c49f6bad8418fc4dec/users/a6c7401eaf744ef895bacc294f7c6268/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.194695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.196326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b65d1d-2399-4691-b5b9-48c31e9cfbc7 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.203766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23fb97c7c4049c49f6bad8418fc4dec, user_id=a6c7401eaf744ef895bacc294f7c6268, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.206932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.208040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1037/2084] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:17:30.300647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1048/2085] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:30 2022] PUT /identity/v3/projects/b23fb97c7c4049c49f6bad8418fc4dec/users/a6c7401eaf744ef895bacc294f7c6268/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:30.305813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.306342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.306741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.333667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:30.334829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.423292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1038/2086] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:30.433167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.436361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.528563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442dff-f17d-4958-b65c-c0785852d97d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.533283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.534073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.534352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.534676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.548911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.550559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.569290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1049/2087] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:30.795599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:30.797103 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:30.809898 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a9f78-c377-4c7c-a50a-795fcad94ab5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:30.819411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:30.820068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:30.820345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:30.820612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] PATH_INFO: `/v3/projects/2e00e84dfa0347ff830c294711d68ff4` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:30.821864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e00e84dfa0347ff830c294711d68ff4)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:30.824129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:30.843774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:30.926375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e00e84dfa0347ff830c294711d68ff4', 'request_id': 'req-84e3fa09-ba82-4f2b-8981-d574b2455ac2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:30.931773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e00e84dfa0347ff830c294711d68ff4', 'request_id': 'req-84e3fa09-ba82-4f2b-8981-d574b2455ac2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:30.939093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e00e84dfa0347ff830c294711d68ff4', 'request_id': 'req-84e3fa09-ba82-4f2b-8981-d574b2455ac2'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.047921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] The token cache is being invalidate because project 2e00e84dfa0347ff830c294711d68ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:31.048116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.048957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.050749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1039/2088] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:30 2022] DELETE /identity/v3/projects/2e00e84dfa0347ff830c294711d68ff4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:31.055746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:31.068256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f43449-6aa9-450c-b12b-08da1cfe4cfa service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:31.131086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:31.131715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:31.131990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:31.132262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] PATH_INFO: `/v3/users/d3ffc23790754f1c942a225ac70b95ff` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:31.133369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ffc23790754f1c942a225ac70b95ff)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:31.136594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:31.368947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ffc23790754f1c942a225ac70b95ff'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.383325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ffc23790754f1c942a225ac70b95ff'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.399293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ffc23790754f1c942a225ac70b95ff'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.403720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:31.550304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ffc23790754f1c942a225ac70b95ff'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.550958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-92e872a4-615b-4520-ae16-f66b6361f814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ffc23790754f1c942a225ac70b95ff'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:31.561438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1050/2089] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:31 2022] DELETE /identity/v3/users/d3ffc23790754f1c942a225ac70b95ff => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:32.260911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:32.262549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:32.287016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fa09-ba82-4f2b-8981-d574b2455ac2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:32.351610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:32.352597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:32.353551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:32.353991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] PATH_INFO: `/v3/projects/5451430eca6c49f38d94d43138acab87` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:32.355359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5451430eca6c49f38d94d43138acab87)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:32.358326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:32.550016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5451430eca6c49f38d94d43138acab87', 'request_id': 'req-e361a0b0-fc16-4f9a-b492-f73c95454fee'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:32.562508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5451430eca6c49f38d94d43138acab87', 'request_id': 'req-e361a0b0-fc16-4f9a-b492-f73c95454fee'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:32.569725 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5451430eca6c49f38d94d43138acab87', 'request_id': 'req-e361a0b0-fc16-4f9a-b492-f73c95454fee'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:32.752830 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] The token cache is being invalidate because project 5451430eca6c49f38d94d43138acab87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:32.753085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:32.754835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e361a0b0-fc16-4f9a-b492-f73c95454fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:32.756365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1040/2090] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:32 2022] DELETE /identity/v3/projects/5451430eca6c49f38d94d43138acab87 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:32.761891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:32.762329 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:32.762507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:32.811342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:32.812964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:32.899081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1051/2091] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:32.904238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:32.904681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:32.905021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:32.935689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:32.938255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:32.950795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1041/2092] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:32.952810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:32.963855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f843-7d92-47cb-807b-27556c9ffa76 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:32.971453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:32.972261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:32.972691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:32.973166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:32.973991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:32.974444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:32.976303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:32.994973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1052/2093] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:32.996148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.008613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5380821-1012-4a20-9728-b8e67e82d308 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.013780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.015066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.015418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.015838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.016616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.021901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:33.139581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1042/2094] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:32 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:33.140287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.150963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc181f7-09e1-4cf4-84ed-50db46c10b39 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.241191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.241832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.242108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.242375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.243042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.247498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:33.342978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1053/2095] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:33 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:33.348348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.362260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:33.368971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99d1b1-e5ac-472b-b2df-0d4b2ba52f5f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.375285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.376018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.376391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.376721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.377496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.379335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:33.394314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1043/2096] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:33.395594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.411273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690273-9741-4b19-851d-0b168e6784ea admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.417595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.418223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.418494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.418798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.419446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.421371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72937cc4-2fd2-4b49-bd0f-4a5a1e62166a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:33.438788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1054/2097] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:33.443163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.469993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2661e0a-5bb2-4b5e-85ad-bc3d8ddfdd91 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.480956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.481994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.482405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.482950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] PATH_INFO: `/v3/projects/b5055dfa0e584c19b419c0d99e9658de/users/db2ebf5ccb3a4dc8824eaadd8c0ccce4/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.509417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5055dfa0e584c19b419c0d99e9658de, user_id=db2ebf5ccb3a4dc8824eaadd8c0ccce4, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.512537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:33.618206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1044/2098] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:33 2022] PUT /identity/v3/projects/b5055dfa0e584c19b419c0d99e9658de/users/db2ebf5ccb3a4dc8824eaadd8c0ccce4/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:33.631131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.631417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.631689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.662988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] MFA Rules not processed for user `db2ebf5ccb3a4dc8824eaadd8c0ccce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:33.664479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:33.736528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:33.761638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1055/2099] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:33.786541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:33.912215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57597-44d1-4cc7-b7d5-6ad14573596d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:33.931082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:33.931674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:33.931956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:33.932354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:33.933046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:33.935243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.033181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1045/2100] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:33 2022] POST /identity/v3/projects => generated 369 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:34.034924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.049453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3413f-47fb-46bd-963c-c0556ec333fb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.126022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.126676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.126984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.127969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.127969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.130626 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.227153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1056/2101] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:34 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:34.231761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.252322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782724-1e35-4f98-ba83-4f96c6230c76 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.257978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.258480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.258870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.259200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.260278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.261919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.294020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1046/2102] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:34.295470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.306027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3abbc-9d6f-47da-87ab-0cd26f7361db admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.316814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.317546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.317832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.318245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] PATH_INFO: `/v3/projects/de4adceae1654bf393779e230fee6712/users/c8017553804f4bb68425133805c2dafe/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.339887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de4adceae1654bf393779e230fee6712, user_id=c8017553804f4bb68425133805c2dafe, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.343105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.409593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1057/2103] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:34 2022] PUT /identity/v3/projects/de4adceae1654bf393779e230fee6712/users/c8017553804f4bb68425133805c2dafe/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:34.421976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.433549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dda9d-a8dd-4257-9aba-a3246fbe562f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.514872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.515642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.515921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.516193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.516831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.518523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ffd70-0e31-495a-88f3-d4232e153e7a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.528769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1047/2104] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:34.533116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.543807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108774b-9a30-4ecf-bb2a-c2007b748010 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.555842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.555842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.555842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.567995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] PATH_INFO: `/v3/projects/de4adceae1654bf393779e230fee6712/users/c8017553804f4bb68425133805c2dafe/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.581425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de4adceae1654bf393779e230fee6712, user_id=c8017553804f4bb68425133805c2dafe, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.590042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.605658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1058/2105] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:34 2022] PUT /identity/v3/projects/de4adceae1654bf393779e230fee6712/users/c8017553804f4bb68425133805c2dafe/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:34.616677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.617100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.617721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.662115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] MFA Rules not processed for user `c8017553804f4bb68425133805c2dafe`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:34.663742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:34.699377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:34.745419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1048/2106] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:34.759453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.761591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:34.827175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489fab7-2401-41e9-8738-4e70da476dc9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:34.831187 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:34.832073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:34.832354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:34.832645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:34.842664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:34.844122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57270849-aa62-4055-a978-d74a9c36e672 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:34.854264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1059/2107] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:34.971462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:34.973145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:34.994497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f14e6-1b64-4a13-9763-ce062314ab52 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:35.006305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:35.006983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:35.007257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:35.007523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:35.091395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:35.093871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:35.125560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1049/2108] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:37.009643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:37.009643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:37.011996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:37.049780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] MFA Rules not processed for user `a6c7401eaf744ef895bacc294f7c6268`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:37.051363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:37.098985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:37.116951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:37.120054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:37.169634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1060/2109] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:37.178287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:37.182136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:37.305571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e50ba4-1f06-4b09-b2a1-cfd9d9579490 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:37.314787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:37.316764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:37.317053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:37.317325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] PATH_INFO: `/v3/users/b37df5d2b38f49d6a0ce62a511c85630` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:37.318801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11084f4-1b23-4616-8652-a5b7ca78589b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:37.318995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37df5d2b38f49d6a0ce62a511c85630)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:37.321050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:37.334236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:37.334903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:37.335179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:37.335452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:37.348950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:37.350775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e3be-16cc-40fb-a9e1-dbdc7b644232 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:37.374976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1061/2110] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:37.530874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37df5d2b38f49d6a0ce62a511c85630'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:37.547004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37df5d2b38f49d6a0ce62a511c85630'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:37.551266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37df5d2b38f49d6a0ce62a511c85630'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:37.551266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:37.557407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37df5d2b38f49d6a0ce62a511c85630'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:37.558166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37df5d2b38f49d6a0ce62a511c85630'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:37.648508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1050/2111] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:37 2022] DELETE /identity/v3/users/b37df5d2b38f49d6a0ce62a511c85630 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:37.661115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:37.661820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:37.662098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:37.687075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:37.689062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:37.773911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1062/2112] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:37.785151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:37.786906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:37.885199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b77151-d758-4dd8-84ad-510c6ae32e66 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:37.894541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:37.895361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:37.896421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:37.896825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:37.909805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:37.911334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30429a1-e95f-4aa4-bcc2-c8221ced2808 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:37.922887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1051/2113] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:38.592978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:38.594134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:38.612061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5259f-8628-4721-b880-76cf17a1d7c3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:38.619404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:38.623469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:38.623469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:38.623469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] PATH_INFO: `/v3/projects/74026e49dc734861b9fd108d303a7b57` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:38.625933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74026e49dc734861b9fd108d303a7b57)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:38.626694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:38.785346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74026e49dc734861b9fd108d303a7b57', 'request_id': 'req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:38.796131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:38.796131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74026e49dc734861b9fd108d303a7b57', 'request_id': 'req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:38.800405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74026e49dc734861b9fd108d303a7b57', 'request_id': 'req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:38.870727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] The token cache is being invalidate because project 74026e49dc734861b9fd108d303a7b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:38.871165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:38.871992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-62ab5ae0-0935-49f1-b201-1b46a6c89bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:38.876031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1063/2114] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:38 2022] DELETE /identity/v3/projects/74026e49dc734861b9fd108d303a7b57 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:38.880839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:38.881386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:38.881386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:38.909183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:38.910059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:38.986760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1052/2115] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:38.994518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:38.994898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:38.995107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.023841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:39.025194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:39.034185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1064/2116] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:39.038698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.048892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62517e9-3bb0-4a9a-9ec5-8fd26f83eeb3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.053814 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.054416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.054699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.054992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.055645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:39.055927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.057551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.073452 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1053/2117] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:39.081992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.095441 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53d0c6-0eda-4ef4-803e-17a2de1eafca None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.100418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.101037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.101364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.101637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.102267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.120275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.247695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1065/2118] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:39 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:39.249200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.262612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cd00-ad61-49a3-bff3-ce3fe3c4483d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.334969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.335208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.340805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.340805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.340805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.340805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.446641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1054/2119] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:39 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:39.451093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.484612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bb4d1-1493-4667-93dc-b7b7c0548465 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.519444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.520447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.523751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.523751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.523751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.525762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.545650 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1066/2120] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:39.547263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.556960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:39.570019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d0155-70a2-4082-b22f-28447dfe0a09 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.572171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.572786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.573062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.573335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.573977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.575718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2839-4a38-4018-af36-838946743c31 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.590380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1055/2121] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:39.595654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.607150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ad9a-1d51-40cb-aa6a-c3d3e001d0be admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:39.618591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:39.619296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.619577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.619847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] PATH_INFO: `/v3/projects/70743271ad9c43d984a0cb907abc7ead/users/c71fbd5777874b98b7106e42f2766fd6/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.645823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70743271ad9c43d984a0cb907abc7ead, user_id=c71fbd5777874b98b7106e42f2766fd6, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:39.654597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:39.762578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1067/2122] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:39 2022] PUT /identity/v3/projects/70743271ad9c43d984a0cb907abc7ead/users/c71fbd5777874b98b7106e42f2766fd6/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:39.777738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8655e96-22c0-42b6-b8dd-87483f698021 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:39.778027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8655e96-22c0-42b6-b8dd-87483f698021 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:39.778306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8655e96-22c0-42b6-b8dd-87483f698021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:39.857312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e8655e96-22c0-42b6-b8dd-87483f698021 None None] MFA Rules not processed for user `c71fbd5777874b98b7106e42f2766fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:39.858550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e8655e96-22c0-42b6-b8dd-87483f698021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:39.950202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1056/2123] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:39.962261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:39.964015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:40.051435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbac6d-63a1-4c03-892d-d13cc2528723 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:40.065387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:40.066725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:40.067309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:40.067778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:40.085298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:40.087314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:40.098672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1068/2124] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:40.510663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e03f-831a-41f0-b7fb-a37b1c0db080 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:40.510988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e03f-831a-41f0-b7fb-a37b1c0db080 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:40.511539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e03f-831a-41f0-b7fb-a37b1c0db080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:40.544111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3051e03f-831a-41f0-b7fb-a37b1c0db080 None None] MFA Rules not processed for user `a6c7401eaf744ef895bacc294f7c6268`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:40.545307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3051e03f-831a-41f0-b7fb-a37b1c0db080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:40.610943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1057/2125] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:41.154865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:41.159038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:41.255184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0163cee-f9f6-4d88-a008-10dc1d808d65 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:41.260051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:41.260705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:41.260985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:41.262149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] PATH_INFO: `/v3/users/db2ebf5ccb3a4dc8824eaadd8c0ccce4` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:41.263441 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2ebf5ccb3a4dc8824eaadd8c0ccce4)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:41.265992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:41.379399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:41.511389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2ebf5ccb3a4dc8824eaadd8c0ccce4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:41.530271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2ebf5ccb3a4dc8824eaadd8c0ccce4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:41.549535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2ebf5ccb3a4dc8824eaadd8c0ccce4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:41.649893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2ebf5ccb3a4dc8824eaadd8c0ccce4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:41.649893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2ebf5ccb3a4dc8824eaadd8c0ccce4'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:41.660822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1069/2126] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:41 2022] DELETE /identity/v3/users/db2ebf5ccb3a4dc8824eaadd8c0ccce4 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:41.665638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:41.666134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:41.666283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:41.703686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:41.706106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:41.803275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1058/2127] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:41.816485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:41.818536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:41.911929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8c75-0551-4f7a-ae4e-1e26d974a608 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:41.915674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:41.916369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:41.916638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:41.916921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:41.930524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:41.932253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:41.946131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1070/2128] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:42.459938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:42.459938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:42.472023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6d0bf-68af-447c-8f29-671033eecbff None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:42.476729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:42.477393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:42.477674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:42.477943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] PATH_INFO: `/v3/projects/b5055dfa0e584c19b419c0d99e9658de` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:42.479184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5055dfa0e584c19b419c0d99e9658de)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:42.481362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:42.508324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=100 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:42.632195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5055dfa0e584c19b419c0d99e9658de', 'request_id': 'req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.638907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5055dfa0e584c19b419c0d99e9658de', 'request_id': 'req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.645543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5055dfa0e584c19b419c0d99e9658de', 'request_id': 'req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.733100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] The token cache is being invalidate because project b5055dfa0e584c19b419c0d99e9658de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:42.734076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.735978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbf1a0a2-094c-4f52-bcfc-43a01bd7500f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.737656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1059/2129] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:42 2022] DELETE /identity/v3/projects/b5055dfa0e584c19b419c0d99e9658de => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:42.741939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:42.752717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b09c3-689f-47a0-9b6b-5cb5b2b46eea service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:42.816386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:42.817216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:42.817491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:42.817755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] PATH_INFO: `/v3/users/c8017553804f4bb68425133805c2dafe` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:42.818931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8017553804f4bb68425133805c2dafe)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:42.820856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:42.895954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def4af-2a29-4d60-b349-a07e7fb0bd90 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:42.896243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def4af-2a29-4d60-b349-a07e7fb0bd90 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:42.896525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def4af-2a29-4d60-b349-a07e7fb0bd90 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:42.897885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1060/2130] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:17:42.907753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:42.908037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:42.908337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:42.928688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] MFA Rules not processed for user `4ef94028a4914a688fd4a4539b1ade66`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:42.930065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:42.993996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8017553804f4bb68425133805c2dafe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:42.999717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:43.015435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8017553804f4bb68425133805c2dafe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.017920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1061/2131] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:17:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:43.022795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:43.026008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:43.050449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea55c0-caa4-4c4a-8d99-0627d5d35040 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:43.055719 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:43.056458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:43.056831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:43.057292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] PATH_INFO: `/v3/users/564805385138452c897fafd5fe976734` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:43.059267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=564805385138452c897fafd5fe976734)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:43.060553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8017553804f4bb68425133805c2dafe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.065875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:43.154723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8017553804f4bb68425133805c2dafe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.155528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8017553804f4bb68425133805c2dafe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.163426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1071/2132] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:42 2022] DELETE /identity/v3/users/c8017553804f4bb68425133805c2dafe => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:43.166285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:43.168090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:43.289284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078262-58d1-4631-8279-aa012de42966 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:43.295873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:43.295873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:43.295873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:43.296254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:43.299203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:43.310861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:43.312767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:43.323930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1072/2133] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:43.330750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564805385138452c897fafd5fe976734'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.346879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564805385138452c897fafd5fe976734'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.347696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564805385138452c897fafd5fe976734'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.358383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564805385138452c897fafd5fe976734'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.360156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564805385138452c897fafd5fe976734'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.470620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1062/2134] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:43 2022] DELETE /identity/v3/users/564805385138452c897fafd5fe976734 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:43.683880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:43.685388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:43.754004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806bcd-f61d-4e47-a1ce-7d8e722dea20 service placement] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:43.798266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:43.800140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:43.815980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:43.816674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:43.821948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:43.821948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] PATH_INFO: `/v3/projects/de4adceae1654bf393779e230fee6712` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:43.821948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de4adceae1654bf393779e230fee6712)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:43.821948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:43.822920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:43.823276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:43.873143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a2f8a-e504-4019-add9-f0baca598b12 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:43.877920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:43.878544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:43.878861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:43.879131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] PATH_INFO: `/v3/projects/bf903b551b1e4fa1b5be2a7ff7db70e7` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:43.880245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf903b551b1e4fa1b5be2a7ff7db70e7)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:43.882146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:43.931726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de4adceae1654bf393779e230fee6712', 'request_id': 'req-a040fe93-e020-4c77-a787-60e253e010f0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.937105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de4adceae1654bf393779e230fee6712', 'request_id': 'req-a040fe93-e020-4c77-a787-60e253e010f0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:43.943464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de4adceae1654bf393779e230fee6712', 'request_id': 'req-a040fe93-e020-4c77-a787-60e253e010f0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.013636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf903b551b1e4fa1b5be2a7ff7db70e7', 'request_id': 'req-84643599-011f-4023-907c-3a69926e9243'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.020031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf903b551b1e4fa1b5be2a7ff7db70e7', 'request_id': 'req-84643599-011f-4023-907c-3a69926e9243'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.026373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf903b551b1e4fa1b5be2a7ff7db70e7', 'request_id': 'req-84643599-011f-4023-907c-3a69926e9243'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.061587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] The token cache is being invalidate because project de4adceae1654bf393779e230fee6712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:44.061781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.063113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a040fe93-e020-4c77-a787-60e253e010f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.064572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1073/2135] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:43 2022] DELETE /identity/v3/projects/de4adceae1654bf393779e230fee6712 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:44.069342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206297c-6255-4c35-81e6-39a80cbaa746 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.069623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206297c-6255-4c35-81e6-39a80cbaa746 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.069891 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206297c-6255-4c35-81e6-39a80cbaa746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.091641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-3206297c-6255-4c35-81e6-39a80cbaa746 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:44.093146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3206297c-6255-4c35-81e6-39a80cbaa746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.157776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1074/2136] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:44.166957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.167697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.167697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.171209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] The token cache is being invalidate because project bf903b551b1e4fa1b5be2a7ff7db70e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:44.171573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.172355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.173974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1063/2137] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:43 2022] DELETE /identity/v3/projects/bf903b551b1e4fa1b5be2a7ff7db70e7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:44.178529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.180078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.190933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:44.192402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.195352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84643599-011f-4023-907c-3a69926e9243 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.200148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.200777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.205817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.205817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] PATH_INFO: `/v3/users/8013e81ccd564efab246c5c7644ea9a3` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.206096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1075/2138] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:44.206178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8013e81ccd564efab246c5c7644ea9a3)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.206178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.206469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.217423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ace5-0d4a-4ec2-87a7-be092ea0b131 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.222888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.223903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.224334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.224609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.225275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:44.225729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.228378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.242296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1076/2139] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:17:44.247084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.258960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c566a1-9617-4559-bc7f-901ce734d68f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.264170 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.264843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.265184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.266274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.267000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.270416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.326060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8013e81ccd564efab246c5c7644ea9a3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.345729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8013e81ccd564efab246c5c7644ea9a3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.346554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8013e81ccd564efab246c5c7644ea9a3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.351133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1077/2140] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:44 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:44.353364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.355454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8013e81ccd564efab246c5c7644ea9a3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.356695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8013e81ccd564efab246c5c7644ea9a3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.428043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05d1f-b9ad-44c5-b7ef-5d78ce69db92 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.433911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.434939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.435215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.435483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.436567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.439032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.440832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1064/2141] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:44 2022] DELETE /identity/v3/users/8013e81ccd564efab246c5c7644ea9a3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:44.556976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:44.571575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1078/2142] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:44 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:44.572217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.574027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.620834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.622605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.628436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:44.635254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c2823-4fbd-49f8-be44-e632f246f06a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.682133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82ec08-f751-498c-bc78-2a9428f478b8 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.688933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.689542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.689820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.690359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.691036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.692879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.706740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1065/2143] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:44.707195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.707822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.708095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.708363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] PATH_INFO: `/v3/users/c71fbd5777874b98b7106e42f2766fd6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.709862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71fbd5777874b98b7106e42f2766fd6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.710566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.711960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.723972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93af6-f7cf-4463-bda2-093d78c23903 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.729139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.730191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.730191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.730487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.731231 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.732964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.745077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1066/2144] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:44.748834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.760114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f38f3-04d8-4d22-b0d1-25271412985a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.765913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.766833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.767116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.767395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] PATH_INFO: `/v3/projects/c95ed3d2c7fc4ef99fd3ac37253b5da2/users/ce9db221645041408afc7ddcb0575f33/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.784009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ed3d2c7fc4ef99fd3ac37253b5da2, user_id=ce9db221645041408afc7ddcb0575f33, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.786673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.887656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1067/2145] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:44 2022] PUT /identity/v3/projects/c95ed3d2c7fc4ef99fd3ac37253b5da2/users/ce9db221645041408afc7ddcb0575f33/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:44.888667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:44.891067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:44.968277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bbe9d-0220-4c7c-a02e-50ee56b8a4a6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:44.973673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:44.974474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:44.974573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71fbd5777874b98b7106e42f2766fd6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.975040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:44.975177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] PATH_INFO: `/v3/projects/4d788ddfb75f4d78863bd0df4e516890` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:44.976267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d788ddfb75f4d78863bd0df4e516890)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:44.978228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:44.982481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71fbd5777874b98b7106e42f2766fd6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:44.996748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71fbd5777874b98b7106e42f2766fd6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.081893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71fbd5777874b98b7106e42f2766fd6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.082599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-941f82f9-1d15-4609-98b1-15b13e2e9d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71fbd5777874b98b7106e42f2766fd6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.089875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1079/2146] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:44 2022] DELETE /identity/v3/users/c71fbd5777874b98b7106e42f2766fd6 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:45.091208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ed9e-d073-406d-9cc5-ec8b9cef93ca None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.092011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ed9e-d073-406d-9cc5-ec8b9cef93ca None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.092301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ed9e-d073-406d-9cc5-ec8b9cef93ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.100209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d788ddfb75f4d78863bd0df4e516890', 'request_id': 'req-d9d29ea2-0259-481d-9e63-c144647e5b78'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.106124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d788ddfb75f4d78863bd0df4e516890', 'request_id': 'req-d9d29ea2-0259-481d-9e63-c144647e5b78'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.113366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d788ddfb75f4d78863bd0df4e516890', 'request_id': 'req-d9d29ea2-0259-481d-9e63-c144647e5b78'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.134557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-79b5ed9e-d073-406d-9cc5-ec8b9cef93ca None None] MFA Rules not processed for user `ce9db221645041408afc7ddcb0575f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:45.135785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-79b5ed9e-d073-406d-9cc5-ec8b9cef93ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.178063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] The token cache is being invalidate because project 4d788ddfb75f4d78863bd0df4e516890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:45.178359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.179043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d9d29ea2-0259-481d-9e63-c144647e5b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:45.181013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1068/2147] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:44 2022] DELETE /identity/v3/projects/4d788ddfb75f4d78863bd0df4e516890 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:45.187148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.188093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.188435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.214992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:45.215084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1080/2148] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:45.215162 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.215251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.215251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.215251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.233733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:45.235555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.277911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1069/2149] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:45.279039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.280291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.295829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1081/2150] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:45.297223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.299287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.301715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0ac28-6192-4101-9339-e48165c0a5f8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.309207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.310005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.310285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.310555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.311224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.313906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.350930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:45.365927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e885d0f-0ae7-4c22-a152-7aa696539e62 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.369722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.370301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.370573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.370920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.381354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.382340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b01da-b86e-48c5-aabe-17b4712aaab8 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.392852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1082/2151] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:45.394002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.394285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.394552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.418533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:45.418742 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1070/2152] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:45.418888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:45.420067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.423156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:45.495948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1083/2153] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:45.502659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.517485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b60b86-9ebd-42ba-bd7b-fa04170938fb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.522377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5088e-7669-4bbb-8773-ae904797619c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.523666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.524443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.525043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.525434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.526914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:45.527323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.528388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.529003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.529280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.529528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.529637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.530307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.538525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.545148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1084/2154] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:45.550138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.564729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781063-9840-4c31-960f-dea66837cc33 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.573745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.574351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.574619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.574920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.575550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.577518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.629306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1071/2155] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:45.632904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.644431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215756c-1165-4b47-8445-b6a8f3bc0b6f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.649873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.650635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.651358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.651358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.652187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.653820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.671134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1072/2156] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:45.673114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.683444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3f34-a3bc-458f-a714-204152e5172e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.688835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.689571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.689855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.690128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] PATH_INFO: `/v3/projects/3a34e4350b67474aa24a71dfd05a5a71/users/7861cbaaf1854f4193eadb16512776ed/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.706380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a34e4350b67474aa24a71dfd05a5a71, user_id=7861cbaaf1854f4193eadb16512776ed, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.709412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.765080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1085/2157] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:45.772311 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.855165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1073/2158] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:45 2022] PUT /identity/v3/projects/3a34e4350b67474aa24a71dfd05a5a71/users/7861cbaaf1854f4193eadb16512776ed/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:45.857056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.887065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bdfd-41d4-4f49-86e3-8ba62ec52b65 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.889541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:45.934730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309218a5-9222-4cd5-aa2a-2742169646eb admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.941168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.941771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.942079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.942350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.943046 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.944726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.956639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1074/2159] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:45.963449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:45.973931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927527-379e-4cf6-addd-7d06fb66189c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:45.978580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.979271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.979515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:45.979996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.979996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] PATH_INFO: `/v3/projects/3a34e4350b67474aa24a71dfd05a5a71/users/7861cbaaf1854f4193eadb16512776ed/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.980176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:45.980382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:45.980776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:45.981395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.982006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a34e4350b67474aa24a71dfd05a5a71, user_id=7861cbaaf1854f4193eadb16512776ed, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:45.983412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.985989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75751-bfc3-42a3-a01f-30ff323c9c55 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:45.992129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:46.001846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1075/2160] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:45 2022] PUT /identity/v3/projects/3a34e4350b67474aa24a71dfd05a5a71/users/7861cbaaf1854f4193eadb16512776ed/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:46.012386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.012685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.012958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.049440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] MFA Rules not processed for user `7861cbaaf1854f4193eadb16512776ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:46.049440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.104769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1086/2161] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:45 2022] POST /identity/v3/users => generated 287 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:46.105652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.107605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.122655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf269b-1b97-4d49-8204-04882b34ff62 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.134894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1076/2162] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:46.134894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.192703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.193409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.193693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.193970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] PATH_INFO: `/v3/projects/70743271ad9c43d984a0cb907abc7ead` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.195157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70743271ad9c43d984a0cb907abc7ead)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.197208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.220708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd9dfb-be43-4b4c-8017-efcf4a8d20ed None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.228274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.228924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.229202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.229473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.230116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.231823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.245577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1077/2163] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:46.246553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.248590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.308252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70743271ad9c43d984a0cb907abc7ead', 'request_id': 'req-e624554b-3f34-46dd-b327-6be4cea6f779'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.315696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70743271ad9c43d984a0cb907abc7ead', 'request_id': 'req-e624554b-3f34-46dd-b327-6be4cea6f779'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.317924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:46.319267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0b6f3-55fc-457d-83c4-a3017952fd0f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.327454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70743271ad9c43d984a0cb907abc7ead', 'request_id': 'req-e624554b-3f34-46dd-b327-6be4cea6f779'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.373149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.373753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.374029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.374299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.424893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] The token cache is being invalidate because project 70743271ad9c43d984a0cb907abc7ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:46.425085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.425715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.427425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1087/2164] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:46 2022] DELETE /identity/v3/projects/70743271ad9c43d984a0cb907abc7ead => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:46.428254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.431944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.433303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.434889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dcf22-470e-41e9-951e-9de3622ee59e service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.445262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1078/2165] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:46.446745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.452329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.452329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.475107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:46.482023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:46.483380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.504060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624554b-3f34-46dd-b327-6be4cea6f779 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.510193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.515402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.515402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.515402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.515402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.515402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.527223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1088/2166] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:46.528222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.529919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.546689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bc27c-9507-41e7-8910-852f3719f0c1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.565255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1079/2167] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:46.565255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.568698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.569541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.569976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.572430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] PATH_INFO: `/v3/users/ce9db221645041408afc7ddcb0575f33` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.572596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba4dd0-c245-4f5f-8cf5-1d920b222629 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.572747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9db221645041408afc7ddcb0575f33)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.575070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.576500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.577331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.578848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.579194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] PATH_INFO: `/v3/projects/6c31ec802fe94d13bb08863101601f9b/users/16ed2e1ac0d24dafb5a945dba1dcbcae/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.600889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c31ec802fe94d13bb08863101601f9b, user_id=16ed2e1ac0d24dafb5a945dba1dcbcae, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.603955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56147f-bed6-4e6c-8bb5-897a7f3a37c7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.686430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1080/2168] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:46 2022] PUT /identity/v3/projects/6c31ec802fe94d13bb08863101601f9b/users/16ed2e1ac0d24dafb5a945dba1dcbcae/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:46.686430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563e995-1bbf-47d6-83cf-3e8cb68b6d0e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.686668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563e995-1bbf-47d6-83cf-3e8cb68b6d0e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.686942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563e995-1bbf-47d6-83cf-3e8cb68b6d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.714154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8563e995-1bbf-47d6-83cf-3e8cb68b6d0e None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:46.715391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8563e995-1bbf-47d6-83cf-3e8cb68b6d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.750602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:46.780893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1081/2169] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:46.782969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.783369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.783739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.786688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9db221645041408afc7ddcb0575f33'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.793476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9db221645041408afc7ddcb0575f33'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.805719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9db221645041408afc7ddcb0575f33'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.811358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] MFA Rules not processed for user `16ed2e1ac0d24dafb5a945dba1dcbcae`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:46.817946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.878394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1082/2170] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:46.879274 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.900767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e52572-6b6e-4373-a468-91a4181fb95d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.900767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.900966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.901062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.901336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.901977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:46.902276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.904257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.917233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1083/2171] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:17:46.918151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.927537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9db221645041408afc7ddcb0575f33'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.928150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9db221645041408afc7ddcb0575f33'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:46.934378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638481b-9315-40ee-882a-7ca49921d952 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:46.939627 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:46.940386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:46.940670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:46.940942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:46.941809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:46.943829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d58f0-05e1-44a1-a3cb-484861c235a2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:46.947662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1089/2172] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:46 2022] DELETE /identity/v3/users/ce9db221645041408afc7ddcb0575f33 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:46.948617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:46.951159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:46.993654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:47.025806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af34f2-327a-41a5-ab8e-375abd69b511 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.030714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.031525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.031803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.032137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.032719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.034697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.104025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1084/2173] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:47.104025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.104025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.104025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.121622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:47.122926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.138219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1090/2174] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:46 2022] POST /identity/v3/projects => generated 369 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:47.138219 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.139603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.196881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1085/2175] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:47.197799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.208674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bf258-5d59-4b6b-abff-a3eed946f1c6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.220612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f83e8-9e78-45e2-afd9-26d3396e721a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.225649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.226356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.226637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.226939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.227554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.229596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.237595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:47.269846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.270555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.270870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.271142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.271753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.273667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.356261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1091/2176] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:47 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:47.357144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.358735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.387067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1086/2177] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:47 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:47.387067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.398003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d91c2-9301-4104-8067-90153400a47c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.402932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.403605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.403950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.404287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.404998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.407581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.418293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1087/2178] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:47.419381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.433345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b70921-54b1-4983-9912-51a210352c85 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.437240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc0108-3399-4111-8842-42cdbafc34b9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.439134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.439780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.440076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.440337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.440962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.441946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.442542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.442660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.442852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.443176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.453514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.455253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.455369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1088/2179] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:47.459248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.466041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1092/2180] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:47.466822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.470486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde2080-0edf-4f00-8970-bae01ba8a302 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.476294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.477153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.477270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5c545-e1f6-4ea7-ae4f-32c4641cc48f service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.477437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.477779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] PATH_INFO: `/v3/projects/67826ece566a41829b99a7f85981ea18/users/b2eca4ed706c47b3b2e0c41d19e82455/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.482084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.482885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.483155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.483425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.484651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.486296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.497247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1093/2181] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:47.502930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67826ece566a41829b99a7f85981ea18, user_id=b2eca4ed706c47b3b2e0c41d19e82455, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.503477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.505813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.513774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148587f6-f97a-4995-a19f-89d639cafdf3 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.519929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.523917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.523917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.524082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] PATH_INFO: `/v3/projects/108fd3e7484c44e78b862ac9473bd272/users/05111c2717534356a6c8c570748ede84/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.541384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=108fd3e7484c44e78b862ac9473bd272, user_id=05111c2717534356a6c8c570748ede84, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.544320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59a5e0-84d9-4eb3-9b6e-0d91f5ebbd3c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.601088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1089/2182] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:47 2022] PUT /identity/v3/projects/67826ece566a41829b99a7f85981ea18/users/b2eca4ed706c47b3b2e0c41d19e82455/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:47.605406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.631143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1094/2183] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:47 2022] PUT /identity/v3/projects/108fd3e7484c44e78b862ac9473bd272/users/05111c2717534356a6c8c570748ede84/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:47.642962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.643254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.643540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.672158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fc8f-d909-426e-bb5f-5e3e6bef237b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.690069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] MFA Rules not processed for user `05111c2717534356a6c8c570748ede84`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:47.692934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.698808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:47.753820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.754918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.755348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.755768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.756741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.758696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.778115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1090/2184] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:47.779705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.793474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7eb3-eed3-4c3b-8a2e-d1cdd917a994 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.805780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1095/2185] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:47.805909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.805909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.805909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.805909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] PATH_INFO: `/v3/projects/67826ece566a41829b99a7f85981ea18/users/b2eca4ed706c47b3b2e0c41d19e82455/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.806225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67826ece566a41829b99a7f85981ea18, user_id=b2eca4ed706c47b3b2e0c41d19e82455, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:47.807630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a50f6-a385-4bf3-9747-11130a5f8397 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:47.817212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:47.825730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1091/2186] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:47 2022] PUT /identity/v3/projects/67826ece566a41829b99a7f85981ea18/users/b2eca4ed706c47b3b2e0c41d19e82455/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:47.826025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.828462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.838897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.839312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.839487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.872576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] MFA Rules not processed for user `b2eca4ed706c47b3b2e0c41d19e82455`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:47.873822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:47.892474 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c39ae2-06fd-4c87-934c-e8b204fa0912 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:47.941445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:47.941962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:47.942303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:47.942574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:47.945066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1092/2187] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:47.960718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:47.972311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:48.014250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:48.016870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:48.031165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1096/2188] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:17:48.033070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:48.034563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:48.059061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:48.066603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0501db-3a8f-48cb-b28f-07f0e03c9101 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:48.072056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:48.072761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:48.073147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:48.073511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] PATH_INFO: `/v3/projects/c95ed3d2c7fc4ef99fd3ac37253b5da2` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:48.075109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95ed3d2c7fc4ef99fd3ac37253b5da2)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:48.077698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:48.144531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3a22-9fef-43db-a399-75a1bc7181b0 service glance] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:48.152818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:48.158928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:48.159568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:48.159941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:48.161908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:48.253470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95ed3d2c7fc4ef99fd3ac37253b5da2', 'request_id': 'req-482d7920-cf99-45d7-a6da-b0b65e50cac9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.260028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:48.261687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:48.264815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95ed3d2c7fc4ef99fd3ac37253b5da2', 'request_id': 'req-482d7920-cf99-45d7-a6da-b0b65e50cac9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.275373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95ed3d2c7fc4ef99fd3ac37253b5da2', 'request_id': 'req-482d7920-cf99-45d7-a6da-b0b65e50cac9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.280616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1097/2189] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:48.385337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] The token cache is being invalidate because project c95ed3d2c7fc4ef99fd3ac37253b5da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:48.385631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.386967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.389308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1093/2190] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:47 2022] DELETE /identity/v3/projects/c95ed3d2c7fc4ef99fd3ac37253b5da2 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:48.393834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:48.413282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3516e3-902f-41eb-b269-f2fc924e63e9 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:48.480473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:48.481098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:48.481376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:48.481645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] PATH_INFO: `/v3/users/7861cbaaf1854f4193eadb16512776ed` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:48.482843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7861cbaaf1854f4193eadb16512776ed)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:48.485767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:48.686583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7861cbaaf1854f4193eadb16512776ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.720684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7861cbaaf1854f4193eadb16512776ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.733310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:48.735568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7861cbaaf1854f4193eadb16512776ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.926233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7861cbaaf1854f4193eadb16512776ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.926880 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9180204a-ba66-4c3b-8c3a-1e2934a0e41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7861cbaaf1854f4193eadb16512776ed'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:48.933093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1098/2191] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:48 2022] DELETE /identity/v3/users/7861cbaaf1854f4193eadb16512776ed => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:49.420335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:49.422778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:49.437831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d7920-cf99-45d7-a6da-b0b65e50cac9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:49.462792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:49.509838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:49.510666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:49.511517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:49.511923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] PATH_INFO: `/v3/projects/3a34e4350b67474aa24a71dfd05a5a71` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:49.513111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a34e4350b67474aa24a71dfd05a5a71)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:49.515292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:49.619747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a34e4350b67474aa24a71dfd05a5a71', 'request_id': 'req-79a66c2f-ad0a-4c2d-be33-dfaa91308475'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:49.625846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a34e4350b67474aa24a71dfd05a5a71', 'request_id': 'req-79a66c2f-ad0a-4c2d-be33-dfaa91308475'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:49.631974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a34e4350b67474aa24a71dfd05a5a71', 'request_id': 'req-79a66c2f-ad0a-4c2d-be33-dfaa91308475'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:49.704976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] The token cache is being invalidate because project 3a34e4350b67474aa24a71dfd05a5a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:49.705290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:49.705903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-79a66c2f-ad0a-4c2d-be33-dfaa91308475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:49.707597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1094/2192] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:49 2022] DELETE /identity/v3/projects/3a34e4350b67474aa24a71dfd05a5a71 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:49.712521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:49.713074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:49.713074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:49.733512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:49.735467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:49.801664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1099/2193] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:49.810682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:49.810994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:49.811271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:49.833781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:49.837431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:49.845107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1095/2194] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:49.849321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:49.864382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a67bfa-b9fd-4f84-bc2a-5e240d10ecb5 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:49.870797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:49.871452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:49.871792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:49.872085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:49.872883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:17:49.873365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:49.875643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:49.890793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1100/2195] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:17:49.895722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:49.909106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6344343-7c3c-4a13-b7cc-9c0e65af18fd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:49.914273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:49.914917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:49.915201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:49.916304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:49.916304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:49.920756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.050310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1096/2196] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:49 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:50.050415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.059009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6b98e-f025-4859-9952-d0c6567569dd admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.130654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.131294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.131566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.131929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.132551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.134493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.243009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1101/2197] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:50 2022] POST /identity/v3/users => generated 300 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:50.246918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.257112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f255df-b28f-4109-9110-8c5efea2f838 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.261884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.262471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.262774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.263448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.263725 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.265312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.274941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1097/2198] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:50.278884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.289323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de47a86-c14b-4b86-908e-b415b1db8a5e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.293911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.294437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.294727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.295002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.295624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.297865 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8e28b-4461-4a7f-b324-59bebc7d6c79 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.308612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1102/2199] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:50.313007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.324486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5401c67-e3c5-43cf-9a4f-9c518863a370 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.329631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.330275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.330549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.330850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] PATH_INFO: `/v3/projects/d4ca41ee3c5c4d898a2093c145b631c5/users/1157b9b206644e36a502b67efb284405/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.345707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ca41ee3c5c4d898a2093c145b631c5, user_id=1157b9b206644e36a502b67efb284405, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.348395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.430699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1098/2200] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:50 2022] PUT /identity/v3/projects/d4ca41ee3c5c4d898a2093c145b631c5/users/1157b9b206644e36a502b67efb284405/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:17:50.441337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.441617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.441887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.475863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] MFA Rules not processed for user `1157b9b206644e36a502b67efb284405`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:50.477059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:50.525628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:50.554026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1103/2201] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:50.558684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.624990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:50.644406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8048d02-cbe1-4060-b1df-c5bba082d7b2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.649354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.649957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.650241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.650512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.651265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.653177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.754163 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1099/2202] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:17:50 2022] POST /identity/v3/projects => generated 401 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:17:50.758532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.772065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e91edb-6a0c-40bf-9adb-0a4495cac4a2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.854914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.855556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.855920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.856393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.857064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.859089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.951996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1104/2203] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:17:50 2022] POST /identity/v3/users => generated 307 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:17:50.956989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:50.967561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef2b15-1daa-4566-a616-20a88802ef87 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:50.997767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:50.997767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:50.997767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:50.997767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:50.998049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:50.998049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:50.998923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1100/2204] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:17:51.003977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:51.019783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da352f6e-de96-4241-8044-e5e7a92429c8 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:51.026520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:51.027500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.027890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.028293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] PATH_INFO: `/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7/users/1be6866f1a444fe29b29fbc9d375ec1a/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:51.052285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9df6b8dacd466c9252bf154a0ea7e7, user_id=1be6866f1a444fe29b29fbc9d375ec1a, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:51.065080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:51.175358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1105/2205] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:51 2022] PUT /identity/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7/users/1be6866f1a444fe29b29fbc9d375ec1a/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:51.179457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:51.206762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9d20-8510-4b46-af77-64ec4c424430 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:51.314602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:51.317249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.317249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.317249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:51.317249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:51.318598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62233-3d25-49b6-acc8-f508b3dbaeab admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:51.340424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1101/2206] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:17:51.344972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:51.356257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f935b3-a2de-4c06-83e7-163b123843cf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:51.363324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:51.364408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.364930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.365372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] PATH_INFO: `/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7/users/1be6866f1a444fe29b29fbc9d375ec1a/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:51.367934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9df6b8dacd466c9252bf154a0ea7e7, user_id=1be6866f1a444fe29b29fbc9d375ec1a, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:51.372963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:51.384117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1106/2207] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:17:51 2022] PUT /identity/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7/users/1be6866f1a444fe29b29fbc9d375ec1a/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:51.395673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.396253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.396786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:51.454665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] MFA Rules not processed for user `1be6866f1a444fe29b29fbc9d375ec1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:51.456900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:51.539318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1102/2208] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:51.595013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:51.596861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:51.663867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bc0c-611f-4cf1-bc11-7d27c984c5bf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:51.668154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:51.668834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.669109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.669378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:51.679484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:51.682326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:51.695588 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1107/2209] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:51.958006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:51.959480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:51.972482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef19b17-ec42-42b9-a150-ea246460e946 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:51.976443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:51.977048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:51.988096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:51.988096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:52.020678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:52.045299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:52.047396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243edb7-e401-4bfa-9185-e87bafae59e4 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:52.056591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1103/2210] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:52.860620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:52.860620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:52.916853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:52.962636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ef6cd-2bbc-4bc9-bffb-6a7747f17054 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:52.969254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:52.970794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:52.971146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:52.971544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] PATH_INFO: `/v3/users/16ed2e1ac0d24dafb5a945dba1dcbcae` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:52.973315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ed2e1ac0d24dafb5a945dba1dcbcae)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:52.975536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:53.130322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ed2e1ac0d24dafb5a945dba1dcbcae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:53.138314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ed2e1ac0d24dafb5a945dba1dcbcae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:53.151747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ed2e1ac0d24dafb5a945dba1dcbcae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:53.224636 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ed2e1ac0d24dafb5a945dba1dcbcae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:53.225246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ed2e1ac0d24dafb5a945dba1dcbcae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:53.234832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1108/2211] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:52 2022] DELETE /identity/v3/users/16ed2e1ac0d24dafb5a945dba1dcbcae => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:53.240354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:53.241276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:53.241725 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:53.263945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:53.265314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:53.354981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1104/2212] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:17:53.368406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:53.373655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:53.471826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78361943-13e4-4d52-8262-eb91c975a354 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:53.475735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:53.476285 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:53.476752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:53.477212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:53.488198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:53.489759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:53.503954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1109/2213] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:17:53.987959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:53.987959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:54.016368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d5961-bfb3-4f7f-96a4-dcd761962def None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:54.026730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:54.027384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:54.027667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:54.027938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] PATH_INFO: `/v3/projects/6c31ec802fe94d13bb08863101601f9b` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:54.029146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c31ec802fe94d13bb08863101601f9b)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:54.031129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:54.144934 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c31ec802fe94d13bb08863101601f9b', 'request_id': 'req-86d65897-34f2-4c48-9096-dba1484d1938'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.150870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c31ec802fe94d13bb08863101601f9b', 'request_id': 'req-86d65897-34f2-4c48-9096-dba1484d1938'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.159264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c31ec802fe94d13bb08863101601f9b', 'request_id': 'req-86d65897-34f2-4c48-9096-dba1484d1938'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.229603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] The token cache is being invalidate because project 6c31ec802fe94d13bb08863101601f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:54.229796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.230415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.231971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1105/2214] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:53 2022] DELETE /identity/v3/projects/6c31ec802fe94d13bb08863101601f9b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:54.236444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:54.248915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cb31c-90a0-454a-83d4-f9616d8a1a1a service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:54.315137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:54.315771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:54.316050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:54.316472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] PATH_INFO: `/v3/users/b2eca4ed706c47b3b2e0c41d19e82455` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:54.317612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2eca4ed706c47b3b2e0c41d19e82455)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:54.319574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:54.321989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:54.510661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2eca4ed706c47b3b2e0c41d19e82455'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.519504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2eca4ed706c47b3b2e0c41d19e82455'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.533836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2eca4ed706c47b3b2e0c41d19e82455'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.613079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2eca4ed706c47b3b2e0c41d19e82455'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.613795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2eca4ed706c47b3b2e0c41d19e82455'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:54.620638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1110/2215] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:54 2022] DELETE /identity/v3/users/b2eca4ed706c47b3b2e0c41d19e82455 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:55.062679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:55.066863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:55.086631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65897-34f2-4c48-9096-dba1484d1938 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:55.152536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:55.153152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:55.153424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:55.153688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] PATH_INFO: `/v3/projects/67826ece566a41829b99a7f85981ea18` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:55.154841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67826ece566a41829b99a7f85981ea18)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:55.156809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:55.158470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:55.275841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67826ece566a41829b99a7f85981ea18', 'request_id': 'req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:55.281318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67826ece566a41829b99a7f85981ea18', 'request_id': 'req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:55.287760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67826ece566a41829b99a7f85981ea18', 'request_id': 'req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:55.355579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] The token cache is being invalidate because project 67826ece566a41829b99a7f85981ea18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:17:55.355879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:55.356505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91f9db5f-d064-4de4-bd4d-cfb5b8a1f3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:55.358002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1106/2216] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:17:55 2022] DELETE /identity/v3/projects/67826ece566a41829b99a7f85981ea18 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:59.478868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:59.480302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:59.494238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880f724-be6d-46ad-9088-4b599b6715b6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:59.556766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:59.557386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:59.557657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:59.557921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] PATH_INFO: `/v3/users/1157b9b206644e36a502b67efb284405` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:59.559083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1157b9b206644e36a502b67efb284405)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:17:59.560983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:17:59.747051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1157b9b206644e36a502b67efb284405'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:59.754484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1157b9b206644e36a502b67efb284405'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:59.767698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1157b9b206644e36a502b67efb284405'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:59.823316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1157b9b206644e36a502b67efb284405'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:59.823961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1157b9b206644e36a502b67efb284405'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:17:59.826392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:17:59.831214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1111/2217] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:17:59 2022] DELETE /identity/v3/users/1157b9b206644e36a502b67efb284405 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:17:59.835855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:59.843647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:59.843647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:17:59.856082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:17:59.857293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:59.921796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1107/2218] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:17:59.932986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:17:59.934472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:17:59.993821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af327-7f11-4232-9c8a-d6bc638cfb12 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:17:59.997511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:17:59.998106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:17:59.998407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:17:59.998676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:00.008030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:00.009394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:00.019502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1112/2219] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:18:00.431403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:00.432977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:00.445346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05877692-994b-4f7e-9adf-5679d238e2f3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:00.449856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:00.450476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:00.450773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:00.451043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] PATH_INFO: `/v3/projects/d4ca41ee3c5c4d898a2093c145b631c5` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:00.452166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ca41ee3c5c4d898a2093c145b631c5)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:00.458782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:00.610356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ca41ee3c5c4d898a2093c145b631c5', 'request_id': 'req-8041c3b7-876f-4a83-a21f-b1fbe13c8262'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:00.617390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ca41ee3c5c4d898a2093c145b631c5', 'request_id': 'req-8041c3b7-876f-4a83-a21f-b1fbe13c8262'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:00.624451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ca41ee3c5c4d898a2093c145b631c5', 'request_id': 'req-8041c3b7-876f-4a83-a21f-b1fbe13c8262'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:00.716151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] The token cache is being invalidate because project d4ca41ee3c5c4d898a2093c145b631c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:00.716524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:00.717198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:00.718775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1108/2220] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:00 2022] DELETE /identity/v3/projects/d4ca41ee3c5c4d898a2093c145b631c5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:00.723084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:00.742271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f6ad1-eac8-4e63-a49b-4824742c8127 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:00.812908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:00.813651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:00.814074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:00.814324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] PATH_INFO: `/v3/users/1be6866f1a444fe29b29fbc9d375ec1a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:00.816806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be6866f1a444fe29b29fbc9d375ec1a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:00.818666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:01.012357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be6866f1a444fe29b29fbc9d375ec1a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:01.020571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be6866f1a444fe29b29fbc9d375ec1a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:01.032970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be6866f1a444fe29b29fbc9d375ec1a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:01.088081 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be6866f1a444fe29b29fbc9d375ec1a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:01.088629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9a310c03-37c6-4ed1-9e4a-cca2f0a94425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be6866f1a444fe29b29fbc9d375ec1a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:01.095126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1113/2221] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:00 2022] DELETE /identity/v3/users/1be6866f1a444fe29b29fbc9d375ec1a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:01.781695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:01.784218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:01.806153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041c3b7-876f-4a83-a21f-b1fbe13c8262 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:01.831500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:01.883810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:01.884462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:01.884743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:01.885020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] PATH_INFO: `/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:01.886878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9df6b8dacd466c9252bf154a0ea7e7)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:01.888565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.020060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9df6b8dacd466c9252bf154a0ea7e7', 'request_id': 'req-464f616a-8b66-4ce7-9513-94d130ced9f3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:02.025334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9df6b8dacd466c9252bf154a0ea7e7', 'request_id': 'req-464f616a-8b66-4ce7-9513-94d130ced9f3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:02.031427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9df6b8dacd466c9252bf154a0ea7e7', 'request_id': 'req-464f616a-8b66-4ce7-9513-94d130ced9f3'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:02.103954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] The token cache is being invalidate because project dc9df6b8dacd466c9252bf154a0ea7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:02.104246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:02.104870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-464f616a-8b66-4ce7-9513-94d130ced9f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:02.106364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1109/2222] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:01 2022] DELETE /identity/v3/projects/dc9df6b8dacd466c9252bf154a0ea7e7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:02.112277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.112976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.112976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.133477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:02.134689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:02.151623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:02.194472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1114/2223] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:02.202488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.202808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.203102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.223317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:02.224511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:02.232530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1110/2224] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:02.237851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.247737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcc5b9-41bb-4a00-af23-59c56a47880d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.252458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.253041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.253317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.253586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.254267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:02.254568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.256213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.268637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1115/2225] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:18:02.273011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.282624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41941b1-0a8c-42e9-85a5-57345229706a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.288033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.288621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.288894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.289159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.289765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.291683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.352724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1111/2226] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:02.357058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.367539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdecf9f-00ca-412c-8288-5c0e6dee6ec0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.428868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.429414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.429721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.429989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.430613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.432591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.589735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1116/2227] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:02 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:18:02.594002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.604954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5c124-c621-4d4e-b172-1cb77f8b5f07 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.609730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.611035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.611319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.611596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.612239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.615324 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.624951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1112/2228] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:02.629911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.639922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432bdb7-bac6-4227-aac9-c2b19f8c3460 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.646047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.647780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.647780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.647780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.647780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.649124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fba73e-8714-4289-bbef-63fae885ace5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.659928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1117/2229] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:02.665014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.675133 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eea7e-5e20-4264-b940-f6b916eb31ca admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.679718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.680323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.680602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.680873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] PATH_INFO: `/v3/projects/5e76fcf3e0c04d23ae47f3a7c885527f/users/b94c5383258c45aa96dae2588d9f9d99/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.697585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e76fcf3e0c04d23ae47f3a7c885527f, user_id=b94c5383258c45aa96dae2588d9f9d99, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.700305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.783333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1113/2230] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:02 2022] PUT /identity/v3/projects/5e76fcf3e0c04d23ae47f3a7c885527f/users/b94c5383258c45aa96dae2588d9f9d99/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:02.794356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.794729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.795014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.822496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] MFA Rules not processed for user `b94c5383258c45aa96dae2588d9f9d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:02.823729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:02.889549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1118/2231] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:02.901193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:02.902940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:02.947863 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:02.962996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb7f2a-37f6-41f8-84fd-bea68d9bb372 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:02.966874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:02.967507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:02.968017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:02.968293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:02.978739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:02.980238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676ebca-db55-4061-a5e6-953b8dd35baf service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:02.993466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1114/2232] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:03.015916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:03.037152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:03.086394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235a0b6-2397-4902-b93a-e883470fcb94 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:03.091105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:03.091721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:03.091991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:03.092255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] PATH_INFO: `/v3/users/b94c5383258c45aa96dae2588d9f9d99` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:03.093387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94c5383258c45aa96dae2588d9f9d99)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:03.095619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:03.244494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94c5383258c45aa96dae2588d9f9d99'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:03.251195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94c5383258c45aa96dae2588d9f9d99'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:03.263365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94c5383258c45aa96dae2588d9f9d99'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:03.310065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94c5383258c45aa96dae2588d9f9d99'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:03.310699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94c5383258c45aa96dae2588d9f9d99'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:03.317479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1119/2233] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:03 2022] DELETE /identity/v3/users/b94c5383258c45aa96dae2588d9f9d99 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:03.321925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:03.322192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:03.322463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:03.341477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:03.342695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:03.401431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1115/2234] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:03.411630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:03.413007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:03.469118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6607129-901f-455f-a2e2-773d1a2a878d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:03.473672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:03.474271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:03.474547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:03.474841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:03.484298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:03.485983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39de76-a59b-4c74-9e6e-75fb473a639f service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:03.496148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1120/2235] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:03.906291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:03.907790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:03.929828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1efac-eff9-4ef8-b114-04f4a9062832 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:03.934966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:03.935629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:03.935901 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:03.936188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] PATH_INFO: `/v3/projects/5e76fcf3e0c04d23ae47f3a7c885527f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:03.937460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e76fcf3e0c04d23ae47f3a7c885527f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:03.939443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.065677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e76fcf3e0c04d23ae47f3a7c885527f', 'request_id': 'req-d57fcdfb-001e-4175-b910-21cb527e2135'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:04.071789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e76fcf3e0c04d23ae47f3a7c885527f', 'request_id': 'req-d57fcdfb-001e-4175-b910-21cb527e2135'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:04.078351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e76fcf3e0c04d23ae47f3a7c885527f', 'request_id': 'req-d57fcdfb-001e-4175-b910-21cb527e2135'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:04.150772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] The token cache is being invalidate because project 5e76fcf3e0c04d23ae47f3a7c885527f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:04.151069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:04.151708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d57fcdfb-001e-4175-b910-21cb527e2135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:04.153252 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1116/2236] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:03 2022] DELETE /identity/v3/projects/5e76fcf3e0c04d23ae47f3a7c885527f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:04.158359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.158646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.159016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.179522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:04.180724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:04.221318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:04.260063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1121/2237] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:04.269069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.269990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.270515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.292020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:04.293490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:04.306014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1117/2238] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:04.310949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.322002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9329912-540f-4c53-a8dd-6c1b8a7167ac None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.327776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.328398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.328697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.328961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.329606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:04.329901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.331527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.349969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1122/2239] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:18:04.351309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.364228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fee903-3069-4b19-bc5f-41b16fe8e3c0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.370831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.371445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.371716 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.371981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.372601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.374507 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.450882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1118/2240] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:04.459840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.479698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051363-229c-44f8-82bd-03f3c3ce6b1b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.548655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.549263 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.549584 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.549881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.551076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.553745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.670331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1123/2241] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:04 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:04.671555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.682781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702eae-bdc6-4f30-a673-5c3b4e5317d2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.687493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.688100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.688378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.688650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.689296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.691028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.707060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1119/2242] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:04.708590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.719944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecb435-f2c7-4e39-99ae-a283d0162ac2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.725786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.726391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.726667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.726964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.727601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.729207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b24649-787b-4cdb-8c98-8a299857bd6f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.742818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1124/2243] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:04.747385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:04.758665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafcbf-c2c6-46ae-be7e-21ddcd159624 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:04.763420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:04.764073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.764351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.764623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] PATH_INFO: `/v3/projects/9b6e5498695b423cae5e1e9e0e82ddd5/users/6c8f714de5a044f9a6d41413d690799a/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.780357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6e5498695b423cae5e1e9e0e82ddd5, user_id=6c8f714de5a044f9a6d41413d690799a, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:04.783222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:04.850290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1120/2244] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:04 2022] PUT /identity/v3/projects/9b6e5498695b423cae5e1e9e0e82ddd5/users/6c8f714de5a044f9a6d41413d690799a/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:04.869632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:04.869915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:04.870214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:04.906261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] MFA Rules not processed for user `6c8f714de5a044f9a6d41413d690799a`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:04.908827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:05.002982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1125/2245] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:05.016515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:05.018233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:05.023699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:05.090967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6c00-fe17-4e9c-8078-65cd6e65eab5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:05.095059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:05.096160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:05.096766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:05.097872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:05.108422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:05.110208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729973e-e6b2-4cc5-82ef-361ad1856813 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:05.120193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1121/2246] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:14.999788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:15.177932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=25 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:19.324274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:19.325748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:19.401071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc0fa7-69a2-4f8e-8c66-467e5592ba1e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:19.405812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:19.406466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:19.406767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:19.407063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] PATH_INFO: `/v3/users/e1d03287e26e4e3dbff823530de33908` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:19.408182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d03287e26e4e3dbff823530de33908)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:19.410111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:19.638875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d03287e26e4e3dbff823530de33908'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:19.646576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d03287e26e4e3dbff823530de33908'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:19.659550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d03287e26e4e3dbff823530de33908'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:19.726810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d03287e26e4e3dbff823530de33908'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:19.727510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d03287e26e4e3dbff823530de33908'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:19.734063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1126/2247] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:19 2022] DELETE /identity/v3/users/e1d03287e26e4e3dbff823530de33908 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:19.739208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:19.740005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:19.740095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:19.759347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:19.765176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:19.819153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1122/2248] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:19.830590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:19.832045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:19.891917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f82c45-3d8c-4ab7-8939-48b89e100dbf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:19.895491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:19.896257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:19.896527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:19.896790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:19.910310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:19.911986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5071d32-1de3-48c1-a614-65471ba3ef52 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:19.923599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1127/2249] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:20.358349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:20.360085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:20.372777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7f4e0-043a-41bd-b431-d5c870a140fe None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:20.385211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:20.385865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:20.386125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:20.386395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] PATH_INFO: `/v3/projects/31ac2cfe44ce4948b3fdc74ba913b8bf` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:20.387598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ac2cfe44ce4948b3fdc74ba913b8bf)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:20.389510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:20.506386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ac2cfe44ce4948b3fdc74ba913b8bf', 'request_id': 'req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:20.512181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ac2cfe44ce4948b3fdc74ba913b8bf', 'request_id': 'req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:20.518477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ac2cfe44ce4948b3fdc74ba913b8bf', 'request_id': 'req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:20.613374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] The token cache is being invalidate because project 31ac2cfe44ce4948b3fdc74ba913b8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:20.613709 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:20.614343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2e0e9c0a-a4e1-420e-aba1-f00df4852e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:20.617142 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1123/2250] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:20 2022] DELETE /identity/v3/projects/31ac2cfe44ce4948b3fdc74ba913b8bf => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:20.621986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:20.622503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:20.622599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:20.642864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:20.644073 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:20.664156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:20.707315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1128/2251] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:20.715256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:20.715533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:20.715800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:20.738581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:20.740410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:20.748979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1124/2252] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:20.753333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:20.763664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238272c6-e0ca-4ce3-9579-1fa96ce7276a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:20.769084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:20.769678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:20.769955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:20.770225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:20.771447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:20.771836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:20.773470 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:20.785298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1129/2253] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:18:20.790352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:20.800329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02559dc-6cb4-4fc6-a9b4-8a9eb38d5237 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:20.806460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:20.807126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:20.807409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:20.807710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:20.808337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:20.810258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:20.923222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1125/2254] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:20 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:20.927858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:20.938503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083edbd0-7045-46fa-8a35-eb745e44b3e0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.003067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.003995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.004277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.004546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.005228 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.007147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.086901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1130/2255] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:20 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:21.091235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.102571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e523919-181b-4266-8fb5-0959dbb16d3e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.107366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.108000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.108278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.108550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.109185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.111083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.121966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1126/2256] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:21.126042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.136155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522703f-b033-4a8e-9efd-61114ab658f5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.140742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.141326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.141596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.141864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.142488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.144120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a936-7a46-4b92-af65-228686bdeec7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.157024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1131/2257] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:21.159809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.170772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d12a7-fd60-49d9-8f13-f473b32a4845 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.175583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.176221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.176505 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.176779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] PATH_INFO: `/v3/projects/d28c9f499bf2475e9c3f9e294c92b027/users/fc5c8268c36f4848adef17f0ffc9dd80/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.192365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28c9f499bf2475e9c3f9e294c92b027, user_id=fc5c8268c36f4848adef17f0ffc9dd80, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.195058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.263849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1127/2258] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:21 2022] PUT /identity/v3/projects/d28c9f499bf2475e9c3f9e294c92b027/users/fc5c8268c36f4848adef17f0ffc9dd80/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:21.274621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.275051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.275335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.303837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] MFA Rules not processed for user `fc5c8268c36f4848adef17f0ffc9dd80`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:21.305053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:21.368021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.371743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:21.378214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1132/2259] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:21.390091 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.391662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:21.406480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:21.423653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:21.450156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777a36-c28c-468b-9461-54a415909eda admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.456446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.456446 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.457082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.457504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] PATH_INFO: `/v3/users/6c8f714de5a044f9a6d41413d690799a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.459556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8f714de5a044f9a6d41413d690799a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.461752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.466096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a49a9-d5a1-44c5-9e3f-aa04935dc5a9 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.469832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.470432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.470719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.471071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.480881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:21.482260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeac763-49f3-4485-8893-50fab2940fe4 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:21.492116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1133/2260] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:18:21.719904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8f714de5a044f9a6d41413d690799a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:21.733000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8f714de5a044f9a6d41413d690799a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:21.733669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8f714de5a044f9a6d41413d690799a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:21.740775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8f714de5a044f9a6d41413d690799a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:21.741423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8f714de5a044f9a6d41413d690799a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:21.820330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1128/2261] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:21 2022] DELETE /identity/v3/users/6c8f714de5a044f9a6d41413d690799a => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:18:21.825105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.825550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.825673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:21.845582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:21.846826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:21.909909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1134/2262] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:21.919619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:21.921007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:21.991818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7b101-ead6-4451-9927-e05dbd5971a4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:21.995665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:21.996287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:21.996566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:21.996829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.006588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:22.008166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:22.019949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1129/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:22.465715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:22.469055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:22.481705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86098a08-feca-4e12-9461-aaaacf319518 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:22.487119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:22.487810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:22.488150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:22.488509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] PATH_INFO: `/v3/users/a400175f70d0495bb2222f51b31279c0` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.491087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a400175f70d0495bb2222f51b31279c0)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:22.493089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:22.591081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:22.592539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:22.607141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ffddd-138d-4dc8-bd9b-bb9a6a1779e7 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:22.617440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:22.618871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:22.618991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:22.619332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] PATH_INFO: `/v3/projects/9b6e5498695b423cae5e1e9e0e82ddd5` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.620615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6e5498695b423cae5e1e9e0e82ddd5)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:22.623087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:22.734078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:22.742150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a400175f70d0495bb2222f51b31279c0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.743894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6e5498695b423cae5e1e9e0e82ddd5', 'request_id': 'req-e66e0188-9e79-47c4-ba25-13f4e5befd7b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.749155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a400175f70d0495bb2222f51b31279c0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.750078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6e5498695b423cae5e1e9e0e82ddd5', 'request_id': 'req-e66e0188-9e79-47c4-ba25-13f4e5befd7b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.756636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6e5498695b423cae5e1e9e0e82ddd5', 'request_id': 'req-e66e0188-9e79-47c4-ba25-13f4e5befd7b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.763562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a400175f70d0495bb2222f51b31279c0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.829791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] The token cache is being invalidate because project 9b6e5498695b423cae5e1e9e0e82ddd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:22.830076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.830696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e66e0188-9e79-47c4-ba25-13f4e5befd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.832225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1130/2264] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:22 2022] DELETE /identity/v3/projects/9b6e5498695b423cae5e1e9e0e82ddd5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:18:22.837443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c023df0-a966-43ea-a344-5e0eb94ea594 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:22.837663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c023df0-a966-43ea-a344-5e0eb94ea594 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:22.837932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c023df0-a966-43ea-a344-5e0eb94ea594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.857331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0c023df0-a966-43ea-a344-5e0eb94ea594 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:22.858514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0c023df0-a966-43ea-a344-5e0eb94ea594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:22.918628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1131/2265] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:22.926170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:22.926449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:22.926731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.937877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a400175f70d0495bb2222f51b31279c0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.938901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-fb9847d6-5bd9-41ba-babf-56a7d0a23912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a400175f70d0495bb2222f51b31279c0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:22.945956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1135/2266] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:22 2022] DELETE /identity/v3/users/a400175f70d0495bb2222f51b31279c0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:22.948265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:22.949468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:22.951092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:22.951485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:22.951769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:22.971264 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:22.972614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:23.007017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:23.015491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1132/2267] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:23.020599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.031078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861a856-83f0-4785-b89c-0bd08003e14b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.035896 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.036484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.036764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.037037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.037681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:23.037983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.039435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1136/2268] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:23.039628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.050249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.051684 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:23.051848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1133/2269] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:18:23.057973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.069947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5cab9-1810-4ef9-8f21-5a653d95896e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.074811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.075680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.076138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.080953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.080953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.080953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.120131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d41ce-e841-42e6-96c0-b80474b0f93f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.126771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.127609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.128075 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.128435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.141778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.144512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.154556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1137/2270] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:18:23.198421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1134/2271] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:23 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:23.199724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.211732 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699335-716f-4c70-a795-2166011e6410 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.297733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.298471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.298941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.299240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.300012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.302036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.432467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1138/2272] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:23 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:18:23.433769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.450194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb886c-b3cf-49fa-8e39-fd24b4a34948 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.460325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.462188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.464510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.465050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.468179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.470846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.488344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1135/2273] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:23.490306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.501182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96495900-0910-4ed4-8194-31833cbe466a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.506939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.510381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.510381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.510381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.510381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.510766 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002adf0-c800-4d27-9e5d-9334cf49898f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.525499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1139/2274] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:23.526829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.539044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8faad5-daa9-48ff-849a-6b3e52980bd9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.544885 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.545789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.546062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.546335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] PATH_INFO: `/v3/projects/cc1aa210813944758fcb1f91cc9775e3/users/93c79dfa4752449790c4d6335318b84e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.565098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1aa210813944758fcb1f91cc9775e3, user_id=93c79dfa4752449790c4d6335318b84e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.568037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.573670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:23.665305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1136/2275] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:23 2022] PUT /identity/v3/projects/cc1aa210813944758fcb1f91cc9775e3/users/93c79dfa4752449790c4d6335318b84e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:23.677939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.678312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.678515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.714735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] MFA Rules not processed for user `93c79dfa4752449790c4d6335318b84e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:23.716252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:23.762974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:23.787344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1140/2276] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:23.799540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:23.800387 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:23.861538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0198bf-ced1-41bd-9968-0f2eccd60490 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:23.866052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:23.866660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:23.866993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:23.867266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:23.878369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:23.879906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3728e44-9073-464e-b199-27d66be7c2fd service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:23.891647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1137/2277] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:24.010644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.012214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:24.088747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eef250-0f8c-4023-a263-c1fb8f23cae1 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.094726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.095423 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.095699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.095970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] PATH_INFO: `/v3/projects/be28bcb0619e4ee78aea88b5123d7a2f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.097095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=be28bcb0619e4ee78aea88b5123d7a2f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.099604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.226071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be28bcb0619e4ee78aea88b5123d7a2f', 'request_id': 'req-bf4cb1b6-5676-4152-8b49-da0956bef2ac'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:24.231391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be28bcb0619e4ee78aea88b5123d7a2f', 'request_id': 'req-bf4cb1b6-5676-4152-8b49-da0956bef2ac'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:24.237603 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be28bcb0619e4ee78aea88b5123d7a2f', 'request_id': 'req-bf4cb1b6-5676-4152-8b49-da0956bef2ac'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:24.305929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] The token cache is being invalidate because project be28bcb0619e4ee78aea88b5123d7a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:24.306242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:24.307016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bf4cb1b6-5676-4152-8b49-da0956bef2ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:24.308714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1141/2278] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:24 2022] DELETE /identity/v3/projects/be28bcb0619e4ee78aea88b5123d7a2f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:24.313479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.313763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.314032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.333755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:24.335349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:24.398069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1138/2279] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:24.406142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.406649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.406956 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.429408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:24.430604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:24.439450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1142/2280] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:24.443903 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.453521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003c058-319c-4d75-aeed-c255517093e7 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.458198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.459325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.459597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.459859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.460502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:24.460796 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.462393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.474955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1139/2281] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:18:24.481118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.490733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b97fd-6166-4196-966d-10ebd67f7138 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.495480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.496069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.496350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.496622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.497243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.499195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.597941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1143/2282] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:24 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:24.602793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.613071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a1ba4-b4f7-4a1a-9cc5-9aa8e10a02da admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.681005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.681949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.681949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.682508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.683002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.685167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.764002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1140/2283] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:24 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:24.765278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.782506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c901e-d0af-4a78-b2d8-30e1660eea8c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.788101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.788815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.789623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.790100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.790890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.794593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.810535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1144/2284] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:18:24.811674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.824769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37189a-6d31-4550-89de-2300aa933af2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.829935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.830670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.830962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.831243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.831881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.834188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c231bbd-6d95-4f9e-a3e6-9c4292ee32dd admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.845448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1141/2285] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:24.854209 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:24.860928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16f0e-7719-4460-8f62-ad00953ce166 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:24.867210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:24.868324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.869138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.869640 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] PATH_INFO: `/v3/projects/1835c316b47c4cb08db14e0eb6e23c97/users/6ee3fa12c3f448a9a419200051a02ab6/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:24.892459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1835c316b47c4cb08db14e0eb6e23c97, user_id=6ee3fa12c3f448a9a419200051a02ab6, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:24.895202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:24.993682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1145/2286] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:24 2022] PUT /identity/v3/projects/1835c316b47c4cb08db14e0eb6e23c97/users/6ee3fa12c3f448a9a419200051a02ab6/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:18:24.998617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14ed83-02d3-44ad-ba10-c7ff3bfbbfc7 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:24.998929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14ed83-02d3-44ad-ba10-c7ff3bfbbfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:24.999200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14ed83-02d3-44ad-ba10-c7ff3bfbbfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:25.008355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:25.019073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-ac14ed83-02d3-44ad-ba10-c7ff3bfbbfc7 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:25.020269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ac14ed83-02d3-44ad-ba10-c7ff3bfbbfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:25.078780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1142/2287] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:25.088957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:25.090893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:25.105467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:25.148151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09cad4-3fa6-44e9-8b8d-884115ad1095 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:25.152082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:25.152676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:25.152948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:25.153211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:25.162044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:25.163886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:25.173260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1146/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:26.353260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6439-4ef6-4a1b-8b08-21f7a96e86d7 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:26.353864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6439-4ef6-4a1b-8b08-21f7a96e86d7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:26.353983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6439-4ef6-4a1b-8b08-21f7a96e86d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:26.382493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a9ac6439-4ef6-4a1b-8b08-21f7a96e86d7 None None] MFA Rules not processed for user `6ee3fa12c3f448a9a419200051a02ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:26.383848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a9ac6439-4ef6-4a1b-8b08-21f7a96e86d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:26.450292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1143/2289] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:26.464211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:26.465879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:26.527994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561da5a-c104-4e6a-a4ce-93ececb6d077 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:26.527994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:26.528462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:26.528735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:26.529000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:26.537007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:26.538527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:26.547316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1147/2290] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:31.405383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:31.406008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:31.406008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:31.424933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] MFA Rules not processed for user `6ee3fa12c3f448a9a419200051a02ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:31.426127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:31.433753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1144/2291] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:35.453832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:35.455293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:35.529966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c863ac2-5942-47a1-9329-15b0f435fb94 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:35.535385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:35.536053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:35.536343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:35.536890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:35.538662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:35.551902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:35.553571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08708a-e081-46d6-a411-81d57093bd12 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:35.564050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1148/2292] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:41.419842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=60 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:45.559748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=79 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:46.667959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:46.670258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:46.688498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6323d-a222-4c6a-af19-c7fcac38480c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:46.693067 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:46.693686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:46.693959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:46.694249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] PATH_INFO: `/v3/users/05111c2717534356a6c8c570748ede84` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:46.695455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=05111c2717534356a6c8c570748ede84)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:46.697435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:46.898086 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05111c2717534356a6c8c570748ede84'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:46.911974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05111c2717534356a6c8c570748ede84'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:46.912829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05111c2717534356a6c8c570748ede84'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:46.921503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05111c2717534356a6c8c570748ede84'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:46.922269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05111c2717534356a6c8c570748ede84'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:47.042620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1145/2293] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:46 2022] DELETE /identity/v3/users/05111c2717534356a6c8c570748ede84 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:47.047726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:47.048268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:47.048268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:47.067469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:47.068692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:47.139963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1149/2294] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:47.150987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:47.152518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:47.213229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5014ec7-1403-402b-8ddc-00e8a31d57fe admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:47.222454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:47.223294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:47.223571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:47.223960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:47.234069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:47.235776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505c7d9-3480-4c63-966b-7bdc7c03f509 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:47.245498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1146/2295] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:47.871026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:47.872503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:47.885190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd7bef-2271-48e0-b90f-c2cce9b5c516 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:47.892025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:47.892666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:47.892944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:47.893210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] PATH_INFO: `/v3/projects/108fd3e7484c44e78b862ac9473bd272` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:47.894530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108fd3e7484c44e78b862ac9473bd272)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:47.896550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:48.043685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108fd3e7484c44e78b862ac9473bd272', 'request_id': 'req-b4551167-05f3-4718-a4d6-fc9ab433e742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:48.049054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108fd3e7484c44e78b862ac9473bd272', 'request_id': 'req-b4551167-05f3-4718-a4d6-fc9ab433e742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:48.055103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108fd3e7484c44e78b862ac9473bd272', 'request_id': 'req-b4551167-05f3-4718-a4d6-fc9ab433e742'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:48.154752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] The token cache is being invalidate because project 108fd3e7484c44e78b862ac9473bd272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:48.155189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:48.156157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:48.157601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1150/2296] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:47 2022] DELETE /identity/v3/projects/108fd3e7484c44e78b862ac9473bd272 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:48.798164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab930f5-c8e1-4153-a180-3c7a33f3edbd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:48.798645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab930f5-c8e1-4153-a180-3c7a33f3edbd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:48.798831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab930f5-c8e1-4153-a180-3c7a33f3edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:48.817708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3ab930f5-c8e1-4153-a180-3c7a33f3edbd None None] MFA Rules not processed for user `18fae6990e4148ff97dec28b204c52df`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:48.818938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3ab930f5-c8e1-4153-a180-3c7a33f3edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:48.873946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1147/2297] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:55.059124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:55.060733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:55.125858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4551167-05f3-4718-a4d6-fc9ab433e742 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:55.131747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:55.132418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:55.132706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:55.132981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] PATH_INFO: `/v3/users/93c79dfa4752449790c4d6335318b84e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:55.134204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c79dfa4752449790c4d6335318b84e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:55.136415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:55.255927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:55.382214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c79dfa4752449790c4d6335318b84e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:55.390761 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c79dfa4752449790c4d6335318b84e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:55.404200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c79dfa4752449790c4d6335318b84e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:55.500560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c79dfa4752449790c4d6335318b84e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:55.501199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c79dfa4752449790c4d6335318b84e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:55.509644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1151/2298] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:55 2022] DELETE /identity/v3/users/93c79dfa4752449790c4d6335318b84e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:55.513108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:55.513560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:55.513749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:55.533897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:55.535151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:55.547593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:55.599807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1148/2299] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:55.609679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:55.609679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:55.670144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7e23-e8ac-4f64-9f9b-4fb3044d1870 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:55.674077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:55.674686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:55.675001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:55.675274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:55.685611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:55.687097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:55.699936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1152/2300] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:56.355199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.357886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.376754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c987b37-8dfd-49af-bf3a-d8859f779721 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:56.380686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.382101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.386242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:56.388878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.389356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.389616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] PATH_INFO: `/v3/projects/cc1aa210813944758fcb1f91cc9775e3` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.392549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1aa210813944758fcb1f91cc9775e3)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:56.395972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:56.407818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902cb36-cd9d-4150-a80f-03548a67e6f8 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:56.415124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:56.415873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.416018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.416290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] PATH_INFO: `/v3/users/fc5c8268c36f4848adef17f0ffc9dd80` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.417995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5c8268c36f4848adef17f0ffc9dd80)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:56.420426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:56.530005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1aa210813944758fcb1f91cc9775e3', 'request_id': 'req-4af2acf4-2715-4565-aee9-b773d28a970b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.535740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1aa210813944758fcb1f91cc9775e3', 'request_id': 'req-4af2acf4-2715-4565-aee9-b773d28a970b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.542368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1aa210813944758fcb1f91cc9775e3', 'request_id': 'req-4af2acf4-2715-4565-aee9-b773d28a970b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.618286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] The token cache is being invalidate because project cc1aa210813944758fcb1f91cc9775e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:56.618580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.619279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-4af2acf4-2715-4565-aee9-b773d28a970b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.621042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1149/2301] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:56 2022] DELETE /identity/v3/projects/cc1aa210813944758fcb1f91cc9775e3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:56.625860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f26c-23c9-491f-a8e8-3f8636f7e979 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.626144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f26c-23c9-491f-a8e8-3f8636f7e979 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.626447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f26c-23c9-491f-a8e8-3f8636f7e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.646174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-86a8f26c-23c9-491f-a8e8-3f8636f7e979 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:56.647617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-86a8f26c-23c9-491f-a8e8-3f8636f7e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.706216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5c8268c36f4848adef17f0ffc9dd80'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.706448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1150/2302] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:56.715082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.715371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.715655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.720160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5c8268c36f4848adef17f0ffc9dd80'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.734955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5c8268c36f4848adef17f0ffc9dd80'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.738102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:56.739912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.747177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1151/2303] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:56.751313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.760310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b755048-fea6-41d1-bc47-86fdbd522b14 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:56.765314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:56.765897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.766175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.766443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.767154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:56.767466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:56.769091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:56.781159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1152/2304] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:18:56.785271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.795495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dfc6b-df88-48f8-a633-cbfc44de03b1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:56.800189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:56.800759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.801037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.801307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.801909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:56.803884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:56.872236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5c8268c36f4848adef17f0ffc9dd80'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.872863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-9d270bea-ba32-4595-8654-e8718a68dca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5c8268c36f4848adef17f0ffc9dd80'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:56.879354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1153/2305] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:18:56 2022] DELETE /identity/v3/users/fc5c8268c36f4848adef17f0ffc9dd80 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:56.883781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.884066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.884416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.911144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1153/2306] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:56 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:18:56.914410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:56.915237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.915945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.917523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.922495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:56.930581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:56.982696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1154/2307] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:18:56.990650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4690-80f6-4dde-b09b-6e0d86f34608 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:56.994222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:56.994733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:56.996862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:56.997461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:56.998107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:56.998107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:56.998648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.000553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.055320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde70794-f789-4085-a991-4c7c266fbdd2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.060165 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.060769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.061046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.061318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.070520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.071981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.082013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1155/2308] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:18:57.126130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1154/2309] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:56 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:57.128172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:57.138398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124bf6-0405-494f-ab25-48c953c84120 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.143117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.143707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.143982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.144619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.146330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.148004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.162037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1156/2310] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:57.163212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:57.174062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8bff-991f-47b8-a89e-592e33f7cc9f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.178909 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.179546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.179829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.180102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.180741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.182363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3b803-7cbc-4ae3-b3fd-1bd7fd2f8a9e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.192494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1155/2311] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:18:57.196683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:57.211074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b131-02be-4af5-a61e-9a352c9e89cc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.216708 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.217343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.217613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.217875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] PATH_INFO: `/v3/projects/b018f573b18c47d89cefbfce67011b28/users/485f1b2575944576b5ddeb1353d1eefb/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.232440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b018f573b18c47d89cefbfce67011b28, user_id=485f1b2575944576b5ddeb1353d1eefb, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.235604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.341252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1157/2312] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:57 2022] PUT /identity/v3/projects/b018f573b18c47d89cefbfce67011b28/users/485f1b2575944576b5ddeb1353d1eefb/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:57.415187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.415479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.415754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.446044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] MFA Rules not processed for user `485f1b2575944576b5ddeb1353d1eefb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:57.447304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:57.514395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1156/2313] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:57.527465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:57.529808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:57.548227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:57.552411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:57.619542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ec0-832b-415b-884c-3a905f73c623 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.624234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.624843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.625111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.625672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.634923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.636831 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f500d-b92e-45f9-b83e-685ba6ca6a56 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.643877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d888e6-6ae1-4b79-a364-e42ac1f29db6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:57.649153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:57.650136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.650420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.650689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] PATH_INFO: `/v3/projects/d28c9f499bf2475e9c3f9e294c92b027` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.652075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28c9f499bf2475e9c3f9e294c92b027)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:57.654288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:57.655060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1158/2314] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:18:57.668871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:57.762427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28c9f499bf2475e9c3f9e294c92b027', 'request_id': 'req-9781828f-012a-46de-8626-8c384b929273'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:57.772300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28c9f499bf2475e9c3f9e294c92b027', 'request_id': 'req-9781828f-012a-46de-8626-8c384b929273'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:57.781333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28c9f499bf2475e9c3f9e294c92b027', 'request_id': 'req-9781828f-012a-46de-8626-8c384b929273'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:57.882151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] The token cache is being invalidate because project d28c9f499bf2475e9c3f9e294c92b027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:18:57.882620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:57.883369 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9781828f-012a-46de-8626-8c384b929273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:18:57.885037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1157/2315] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:18:57 2022] DELETE /identity/v3/projects/d28c9f499bf2475e9c3f9e294c92b027 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:18:57.890431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.890727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.891022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:57.913289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:57.914490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:57.940106 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:18:57.975245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1159/2316] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:57.984804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:57.985483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:57.985940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.019769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:58.020967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:58.028983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1158/2317] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:58.037425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.043173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb3a54-37dc-4997-b50c-3d2ebcb85134 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.048713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.049315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.049590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.049859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.050507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:18:58.050826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.052417 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.064591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1160/2318] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:18:58.068976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.080265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ddc44-bb8b-419e-8a0e-74f1956df615 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.085412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.086018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.086310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.086580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.087249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.089232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.162608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1159/2319] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:18:58 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:58.167459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.177947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367fef2-e001-46f8-b19e-7404d981ff52 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.236021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.236654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.236942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.237234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.237897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.240035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.317401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1161/2320] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:18:58 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:18:58.321844 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.334355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadfda0-bc4f-405a-9809-f368cf4cfbd9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.340213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.340892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.341237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.341598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.342309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.344129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.355769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1160/2321] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:58.364905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.375054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6b94-f43f-4c89-9936-a1e940d5f373 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.379742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.380426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.380741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.381009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.386239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.386239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffce6c-f3d3-4c7f-922f-106db86e1264 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.398153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1162/2322] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:18:58.399406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.412256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055494c-ca94-49d3-96a8-7fb5df2a02d2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.418179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.418817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.419152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.419481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] PATH_INFO: `/v3/projects/ab313819a4b141fc8391e750896f8b33/users/46f8e315028640a7a2106c23018bf158/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.435918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab313819a4b141fc8391e750896f8b33, user_id=46f8e315028640a7a2106c23018bf158, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.438549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.500448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1161/2323] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:18:58 2022] PUT /identity/v3/projects/ab313819a4b141fc8391e750896f8b33/users/46f8e315028640a7a2106c23018bf158/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:18:58.505589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c20e5-8717-47dc-8bdf-caa54f96e425 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.505871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c20e5-8717-47dc-8bdf-caa54f96e425 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.506138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c20e5-8717-47dc-8bdf-caa54f96e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.525431 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-f69c20e5-8717-47dc-8bdf-caa54f96e425 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:18:58.527465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f69c20e5-8717-47dc-8bdf-caa54f96e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:58.587033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1163/2324] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:18:58.597615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:18:58.599453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:18:58.657588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f95b9-84d3-4cc6-a0bb-f34010eb282f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:18:58.661316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:18:58.661910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:18:58.664825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:18:58.664825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:18:58.671749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:18:58.673119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:18:58.683091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1162/2325] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:19:04.237242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.237717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.237812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.269725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] MFA Rules not processed for user `46f8e315028640a7a2106c23018bf158`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:19:04.270894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:04.298360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:04.343257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1164/2326] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:19:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:19:04.347397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.357740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42f6b-3b11-4333-bd19-313ec1369def service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.363362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.364054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.364398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.364733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.365547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.367813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.448809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1163/2327] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:19:04 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:19:04.450514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.462070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57af4e-4e89-4584-9621-6d61987ffbee None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.519421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.520043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.520318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.520582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.521186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.523188 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.626558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1165/2328] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:19:04 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:19:04.630670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.642314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324836-306e-4fba-99e9-a95fd78dd1d6 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.646867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.647482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.647757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.648023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.648649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.650218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.663238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1164/2329] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:19:04.666143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.677260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e2b99-6e7d-48b7-aedb-be00bcfa6483 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.681962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.682628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.682940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.683217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] PATH_INFO: `/v3/projects/67a364027eec4fa3b5d8fc39cda46592/users/0bc4e960be8e4445b2ffe4254d337ae0/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.698988 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a364027eec4fa3b5d8fc39cda46592, user_id=0bc4e960be8e4445b2ffe4254d337ae0, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.701785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.765378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1166/2330] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:19:04 2022] PUT /identity/v3/projects/67a364027eec4fa3b5d8fc39cda46592/users/0bc4e960be8e4445b2ffe4254d337ae0/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:19:04.769596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.780228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348397a3-037f-41e1-b088-76ed3f2d2a67 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.783798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:04.840824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.841438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.841714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.841984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.842651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.844306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452470e6-9e73-494f-a0fd-6ea58e0c3cd6 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.855608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1165/2331] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:19:04.859714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:04.871365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde8bc-826b-41d6-8c3d-3bebe19c7d4c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:04.876312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:04.877023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:04.877248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:04.877521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] PATH_INFO: `/v3/projects/67a364027eec4fa3b5d8fc39cda46592/users/0bc4e960be8e4445b2ffe4254d337ae0/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:04.879469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a364027eec4fa3b5d8fc39cda46592, user_id=0bc4e960be8e4445b2ffe4254d337ae0, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:04.882148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:04.894997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1167/2332] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:19:04 2022] PUT /identity/v3/projects/67a364027eec4fa3b5d8fc39cda46592/users/0bc4e960be8e4445b2ffe4254d337ae0/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:19:10.814346 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84da51-d1c6-4608-829f-4c8f2ccc4c6d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:10.817515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84da51-d1c6-4608-829f-4c8f2ccc4c6d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:10.817806 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84da51-d1c6-4608-829f-4c8f2ccc4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:10.864013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-6d84da51-d1c6-4608-829f-4c8f2ccc4c6d None None] MFA Rules not processed for user `0bc4e960be8e4445b2ffe4254d337ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:19:10.865642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6d84da51-d1c6-4608-829f-4c8f2ccc4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:10.955379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1166/2333] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:19:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:19:10.970422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:10.976315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:11.057992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e40fb-c0c6-4363-93c4-07bb60b343ce admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:11.063951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:11.064569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:11.064863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:11.065129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:11.079778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:11.126544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:11.128304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:11.138241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1168/2334] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:19:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:19:13.609379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5fb2f-2c9b-41d0-a81a-f7cabc5656b8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:13.609890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5fb2f-2c9b-41d0-a81a-f7cabc5656b8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:13.609990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5fb2f-2c9b-41d0-a81a-f7cabc5656b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:13.631140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c1b5fb2f-2c9b-41d0-a81a-f7cabc5656b8 None None] MFA Rules not processed for user `46f8e315028640a7a2106c23018bf158`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:19:13.631140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c1b5fb2f-2c9b-41d0-a81a-f7cabc5656b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:13.638289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1167/2335] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:19:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:19:21.143951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=21 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:23.623692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=63 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:30.230795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:30.232204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:30.302205 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e98e0-5991-4da9-b0f5-6b8510c3c202 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:30.307721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:30.308658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:30.308824 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:30.309101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] PATH_INFO: `/v3/users/485f1b2575944576b5ddeb1353d1eefb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:30.310237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485f1b2575944576b5ddeb1353d1eefb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:30.312300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:30.513002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485f1b2575944576b5ddeb1353d1eefb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:30.519868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485f1b2575944576b5ddeb1353d1eefb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:30.532569 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485f1b2575944576b5ddeb1353d1eefb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:30.624145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485f1b2575944576b5ddeb1353d1eefb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:30.624774 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485f1b2575944576b5ddeb1353d1eefb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:30.632125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1169/2336] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:19:30 2022] DELETE /identity/v3/users/485f1b2575944576b5ddeb1353d1eefb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:19:30.636588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:30.637079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:30.637174 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:30.655788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:19:30.656993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:30.718856 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1168/2337] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:19:30.729021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:30.730426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:30.790137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7c92-5948-41ab-98e1-e76c14f17fd0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:30.793837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:30.794445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:30.794742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:30.795015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:30.805042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:30.806479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:30.815807 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1170/2338] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:19:31.260573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:19:31.262473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:19:31.274476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d45d68-22db-4e17-bd7b-50b3d6b7fa8c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:19:31.281048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:19:31.282433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:19:31.282731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:19:31.283026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] PATH_INFO: `/v3/projects/b018f573b18c47d89cefbfce67011b28` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:19:31.284316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b018f573b18c47d89cefbfce67011b28)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:19:31.286790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:19:31.415379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b018f573b18c47d89cefbfce67011b28', 'request_id': 'req-5898cb3f-9185-40af-8641-64e876461c2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:31.421014 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b018f573b18c47d89cefbfce67011b28', 'request_id': 'req-5898cb3f-9185-40af-8641-64e876461c2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:31.429967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b018f573b18c47d89cefbfce67011b28', 'request_id': 'req-5898cb3f-9185-40af-8641-64e876461c2e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:31.528316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] The token cache is being invalidate because project b018f573b18c47d89cefbfce67011b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:19:31.528529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:31.529153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:19:31.530656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1169/2339] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:19:31 2022] DELETE /identity/v3/projects/b018f573b18c47d89cefbfce67011b28 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:19:40.811706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:19:41.524556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:20:20.764496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:20.773134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b3896-dc0e-47e1-98fc-7ad9a446da0f service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:20.821829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:20.822424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:20.822723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:20.823001 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:20.824910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:20.898030 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:20.900272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:20.911004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1171/2340] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:22.730342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:22.731798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:22.745175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898cb3f-9185-40af-8641-64e876461c2e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:22.748754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:22.749349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:22.749630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:22.749907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:22.751596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:22.765295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:22.766697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfcca6-edad-4772-bbb5-cdbf2a1027e1 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:22.778204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1170/2341] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:23.093453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:23.103768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816c01-eaf2-4ceb-b63f-c86628b91142 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:23.108333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:23.108922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:23.109194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:23.109480 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:23.111502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:23.177646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:23.179328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:23.189042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1172/2342] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:23.442060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f192625-94de-4c32-b36b-d99bb6aa4bfa None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:23.442760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f192625-94de-4c32-b36b-d99bb6aa4bfa None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:23.444033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f192625-94de-4c32-b36b-d99bb6aa4bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:23.482754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-6f192625-94de-4c32-b36b-d99bb6aa4bfa None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:20:23.482754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6f192625-94de-4c32-b36b-d99bb6aa4bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:23.544057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1171/2343] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:20:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:20:23.606552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:23.608054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:23.655175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:20:23.688380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741e7a-7449-4700-b622-695e2aeff3c2 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:23.693810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:23.694434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:23.694756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:23.695029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:23.705084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:23.706896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:23.716845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1173/2344] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:20:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:20:30.406309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:30.406886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:30.406886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:30.425436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] MFA Rules not processed for user `6ee3fa12c3f448a9a419200051a02ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:20:30.426652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:30.489366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1172/2345] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:20:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:20:33.761469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=16 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:20:36.223276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:36.224712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:36.242289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f1b73-f9ea-4f7b-8a29-b98cf3ec418f service swift] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:36.265237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:36.265237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:36.265237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:36.265237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:36.266088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:36.314967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:36.316968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:36.347739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:36.349458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:36.360121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1174/2346] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:36.393877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df350f-40f0-4a1b-95c9-761c3d32cdc5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:36.397477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:36.398085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:36.398364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:36.398624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:36.409242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:36.410730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce3f37-efb8-44cb-960f-60bf75feb806 service glance] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:36.420976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1173/2347] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:44.713789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:44.723878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb8b1-4fe1-470c-9ac1-2d5f70b8db2f service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:44.727532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:44.728124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:44.728401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:44.728671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:44.730388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:44.741182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:44.742607 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c7e47-d805-4380-8e27-847dde70205d service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:44.751717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1175/2348] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:45.212120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc29df-1348-464d-8a5f-2e9c51241bac None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.212690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc29df-1348-464d-8a5f-2e9c51241bac None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.212690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc29df-1348-464d-8a5f-2e9c51241bac None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.214136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1174/2349] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:20:45.221592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c950a-f660-42f5-a370-cac696f94cc5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.222487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c950a-f660-42f5-a370-cac696f94cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.222864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c950a-f660-42f5-a370-cac696f94cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.243902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-958c950a-f660-42f5-a370-cac696f94cc5 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:20:45.245161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-958c950a-f660-42f5-a370-cac696f94cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:45.257156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1176/2350] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:20:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:20:45.269489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc29df-1348-464d-8a5f-2e9c51241bac None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:45.281314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc29df-1348-464d-8a5f-2e9c51241bac None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:45.285132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:45.285721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.286020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.286286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.296624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:45.298045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eb939-2a29-4fca-be64-3f34dc683039 service glance] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:45.308846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1175/2351] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:45.554131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f094c0-98b9-4552-bd5f-4bf76724a798 None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.554445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f094c0-98b9-4552-bd5f-4bf76724a798 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.554730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f094c0-98b9-4552-bd5f-4bf76724a798 None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.556071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1177/2352] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:20:45.561047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3da73-b06b-4d62-8acc-95654db8e17b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.561333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3da73-b06b-4d62-8acc-95654db8e17b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.561608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3da73-b06b-4d62-8acc-95654db8e17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.582698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0ad3da73-b06b-4d62-8acc-95654db8e17b None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:20:45.583866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0ad3da73-b06b-4d62-8acc-95654db8e17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:45.591303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1176/2353] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:20:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:20:45.603366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f094c0-98b9-4552-bd5f-4bf76724a798 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:45.614505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f094c0-98b9-4552-bd5f-4bf76724a798 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:45.618084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:45.618679 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.618981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.619249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.627332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:45.628760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562c180-c284-4a46-bb9d-1e3c5455a6f6 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:45.636721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1178/2354] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:45.720880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7d87-64a0-4090-ac5e-4d57a5d136cf None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.721188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7d87-64a0-4090-ac5e-4d57a5d136cf None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.721460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7d87-64a0-4090-ac5e-4d57a5d136cf None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.722990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1177/2355] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:20:45.729159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.729638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.729767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.749742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:20:45.751019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:45.760131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1179/2356] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:20:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:20:45.771583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7d87-64a0-4090-ac5e-4d57a5d136cf None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:45.782679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7d87-64a0-4090-ac5e-4d57a5d136cf None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:45.786263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:45.786886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.787164 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.787431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.798160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:45.799609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:45.810829 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1178/2357] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:45.877394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:45.883267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:45.946822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a82fa-b3fc-4031-a886-e85425e5a04d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:45.952818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:45.953424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:45.953707 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:45.953977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:45.955768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:45.967206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:45.968780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eaefd2-3194-444e-bb3c-2cedb846f025 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:45.978527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1180/2358] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:55.686373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:20:55.690726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=98 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:20:55.695653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a8d9c-f38a-4185-aaf8-c1cd23e2d3d3 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:20:55.699203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:20:55.700408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:20:55.700791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:20:55.701200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:20:55.703244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:20:55.713493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:20:55.714953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:20:55.724850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1179/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:20:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:20:55.974499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=78 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:00.609869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:00.611922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:00.611922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:00.633936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:21:00.633936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:00.638353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1181/2360] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:21:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:21:00.650278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:00.662887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42829-8f8e-40e9-b57d-d9fd68835b6f service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:00.667774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:00.668357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:00.668630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:00.669687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:00.677850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:00.679937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:00.687551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1180/2361] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:21:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:21:10.624547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:10.684122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=11 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:25.482371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:25.483976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:25.500538 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11a682-44a8-47a6-8369-5ebadbaebc9f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:25.506115 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:25.506796 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:25.507074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:25.507340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:25.509023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:25.521279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:25.523476 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffb463-c30e-4e41-98d5-3648c89c910c service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:25.533934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1182/2362] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:21:33.978197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:33.991509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f2d9-4bef-4632-9c74-fb68c14cf6bb service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:33.997003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:33.997697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:33.997973 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:33.998239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:34.000089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:34.013008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:34.014632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:34.026100 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1181/2363] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:21:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:21:35.529509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:36.881795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:36.882387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:36.888793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:36.902689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:21:36.903970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:36.911958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1183/2364] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:21:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:21:36.938676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:36.950745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c419c-dd58-40c4-8b64-b3eed87ef4c6 service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:36.955862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:36.966667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:36.966667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:36.966667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:36.966667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:36.967138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:36.974902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1182/2365] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:21:46.897814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:46.971550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=12 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:56.247757 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:56.249297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:56.317247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5526e5-faf4-4ba2-b995-1fe9d0fcd152 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:56.322759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:56.323376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:56.323648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:56.323912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] PATH_INFO: `/v3/users/6ee3fa12c3f448a9a419200051a02ab6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:56.325022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee3fa12c3f448a9a419200051a02ab6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:56.326983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:56.492266 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee3fa12c3f448a9a419200051a02ab6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:56.499117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee3fa12c3f448a9a419200051a02ab6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:56.511582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee3fa12c3f448a9a419200051a02ab6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:56.580978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee3fa12c3f448a9a419200051a02ab6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:56.581549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-14df83fe-664c-4f92-854f-8eefc9ed8794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee3fa12c3f448a9a419200051a02ab6'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:56.587896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1184/2366] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:21:56 2022] DELETE /identity/v3/users/6ee3fa12c3f448a9a419200051a02ab6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:21:56.840124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:56.851602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803ead-abc5-46c0-a066-fd85284b81ef service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:56.908376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:56.908993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:56.909268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:56.909535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] PATH_INFO: `/v3/projects/1835c316b47c4cb08db14e0eb6e23c97` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:56.910680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1835c316b47c4cb08db14e0eb6e23c97)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:56.912690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.009485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1835c316b47c4cb08db14e0eb6e23c97', 'request_id': 'req-344940aa-eb11-4f36-a6ca-6d420ee9d737'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:57.015008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1835c316b47c4cb08db14e0eb6e23c97', 'request_id': 'req-344940aa-eb11-4f36-a6ca-6d420ee9d737'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:57.021130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1835c316b47c4cb08db14e0eb6e23c97', 'request_id': 'req-344940aa-eb11-4f36-a6ca-6d420ee9d737'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:57.106626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] The token cache is being invalidate because project 1835c316b47c4cb08db14e0eb6e23c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:21:57.106869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:57.107503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-344940aa-eb11-4f36-a6ca-6d420ee9d737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:21:57.109085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1183/2367] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:21:56 2022] DELETE /identity/v3/projects/1835c316b47c4cb08db14e0eb6e23c97 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:21:57.114293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.114677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.115003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.134576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:21:57.135814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:57.195853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1185/2368] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:21:57.203717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.204003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.204278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.224157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:21:57.225398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:57.233688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1184/2369] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:21:57.238079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.247694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca2ac-4ca3-4d74-a1eb-429447e0a994 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.252296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.252862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.253141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.253408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.254040 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:21:57.254336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.255958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.267260 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1186/2370] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:21:57.271407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.281357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee35eb-81f2-4c30-83fe-88a90a40c7e5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.285948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.286534 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.286848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.287117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.287747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.290308 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.374008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1185/2371] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:21:57 2022] POST /identity/v3/projects => generated 399 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:21:57.378111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.388673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942baa19-dd02-4423-a6f1-ad05231dbc69 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.392189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:57.446805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.447405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.447674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.447939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.448540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.450438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.542112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1187/2372] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:21:57 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:21:57.546287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.557715 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20f02-70ad-4685-b2d2-6f8af2600237 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.562263 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.562905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.563184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.563455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.564091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.565699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.577590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1186/2373] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:21:57.581676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.592369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849d3a-0c71-49d5-8e4e-569b4e813db7 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.597036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.597616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.597888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.598151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.598804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.600870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f75038-b85b-4cc5-ab05-eca277cf0ab8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.611161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1188/2374] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:21:57.615406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.626377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7386e-e5aa-4eae-b235-696bbec58fa2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.630926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.631558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.632058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.632386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] PATH_INFO: `/v3/projects/a0251efff601476eb3e77b9fc1869b2c/users/e91c09bf171146eaa17b2d3713351eba/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.646623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0251efff601476eb3e77b9fc1869b2c, user_id=e91c09bf171146eaa17b2d3713351eba, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.649487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.735431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1187/2375] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:21:57 2022] PUT /identity/v3/projects/a0251efff601476eb3e77b9fc1869b2c/users/e91c09bf171146eaa17b2d3713351eba/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:21:57.746882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.747173 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.747449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.775823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] MFA Rules not processed for user `e91c09bf171146eaa17b2d3713351eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:21:57.777031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:57.839947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1189/2376] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:21:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:21:57.851990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:57.853704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:57.884430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:57.911677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0454c-ba5d-4a91-89d1-a24610b2db30 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:57.917355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:57.917959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:57.918238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:57.918511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:57.927007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:57.928610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:57.938087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1188/2377] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:21:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:21:58.940460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:21:58.941933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:59.015870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc83bd-9459-4f36-878b-84fc8ac0081c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:21:59.021279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:21:59.021877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:21:59.022211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:21:59.022496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:21:59.024273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:21:59.034459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:21:59.082660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:21:59.084284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93a874-8b66-4948-a4fd-9b67963b1440 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:21:59.093137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1190/2378] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:21:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:22:07.882349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:07.947543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7ed4-1b06-4d6e-a25d-02ae146bcf7d service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:07.952147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:07.952779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:07.953072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:07.953344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] PATH_INFO: `/v3/users/e91c09bf171146eaa17b2d3713351eba` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:07.954440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91c09bf171146eaa17b2d3713351eba)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:07.956396 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:08.111601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91c09bf171146eaa17b2d3713351eba'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.124497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91c09bf171146eaa17b2d3713351eba'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.128858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91c09bf171146eaa17b2d3713351eba'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.132159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91c09bf171146eaa17b2d3713351eba'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.132827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91c09bf171146eaa17b2d3713351eba'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.209342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1189/2379] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:22:07 2022] DELETE /identity/v3/users/e91c09bf171146eaa17b2d3713351eba => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:22:08.214116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:08.214598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:08.214749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:08.233778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:08.234996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:08.294275 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1191/2380] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:08.303912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:08.305318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:08.363140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:08.367333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78112b6a-7d40-4563-9625-bd95fe4f955d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:08.371669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:08.372272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:08.372550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:08.372818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:08.383332 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:08.385842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe1ce4-9d11-4195-91f4-96c6a9bdb1d1 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:08.395536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1190/2381] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:22:08.837394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:08.839222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:08.852912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfdde9-db70-4450-97fa-2643b03189ef None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:08.857750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:08.858405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:08.858666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:08.858964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] PATH_INFO: `/v3/projects/a0251efff601476eb3e77b9fc1869b2c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:08.860306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0251efff601476eb3e77b9fc1869b2c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:08.863101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:08.985139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0251efff601476eb3e77b9fc1869b2c', 'request_id': 'req-1cc93305-a13d-4e2d-a9bc-5ad55a920817'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.991434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0251efff601476eb3e77b9fc1869b2c', 'request_id': 'req-1cc93305-a13d-4e2d-a9bc-5ad55a920817'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:08.997866 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0251efff601476eb3e77b9fc1869b2c', 'request_id': 'req-1cc93305-a13d-4e2d-a9bc-5ad55a920817'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:09.086380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] The token cache is being invalidate because project a0251efff601476eb3e77b9fc1869b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:22:09.087239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:09.088093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1cc93305-a13d-4e2d-a9bc-5ad55a920817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:22:09.089612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1192/2382] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:22:08 2022] DELETE /identity/v3/projects/a0251efff601476eb3e77b9fc1869b2c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:22:09.094372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.095138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.095138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.119420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:09.120678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:09.193228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1191/2383] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:09.263315 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.263841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.264272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.296896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:09.299046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:09.316035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1193/2384] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:09.317385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.328800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8b59f-b7d2-4424-8d5d-fe72ea541be2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.333984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.334592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.334906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.335184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.335834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:22:09.336137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.339850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.350745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1192/2385] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:22:09.355237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.365797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bae80-3d1b-41b6-bdd7-6701bcc67a33 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.371722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.372351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.372815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.373139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.373791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.376183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.484413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1194/2386] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:22:09 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:22:09.485761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.496821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eb7d-3827-48f0-bf36-bd5dc04a8970 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.561391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.562192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.569391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.569391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.569391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.569391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.651070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1193/2387] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:22:09 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:22:09.656635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.671302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b722656-da2c-42b3-a3db-9a2a1719bd9a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.676497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.677124 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.677401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.677670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.678314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.679978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.681449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:09.690619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1195/2388] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:22:09.695415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.706039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea79bd-3c11-4745-b995-b5896669a3f5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.713000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.713000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.713000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.713000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.713943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.715280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f1781-4334-449a-a2a4-1eb5a4c94af5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.726435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1194/2389] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:22:09.732157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.747572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610b5bb-727e-4408-a03b-c8573b88f2a3 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:09.752751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:09.753401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.753677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.753948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] PATH_INFO: `/v3/projects/d793302181944f85ade371ded12ca4c4/users/5f4eeae86db248a09f0725ca23ac6a9f/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.769967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d793302181944f85ade371ded12ca4c4, user_id=5f4eeae86db248a09f0725ca23ac6a9f, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:09.776144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:09.853464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1196/2390] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:22:09 2022] PUT /identity/v3/projects/d793302181944f85ade371ded12ca4c4/users/5f4eeae86db248a09f0725ca23ac6a9f/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:22:09.858694 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5f42-da1d-4a8d-a8e4-61657c7de2bf None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:09.859008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5f42-da1d-4a8d-a8e4-61657c7de2bf None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:09.859281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5f42-da1d-4a8d-a8e4-61657c7de2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:09.880586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3bec5f42-da1d-4a8d-a8e4-61657c7de2bf None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:09.881972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3bec5f42-da1d-4a8d-a8e4-61657c7de2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:09.937827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:09.950029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1195/2391] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:09.963629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:09.965399 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:10.028833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860141-743f-4d02-897f-274f669dccb1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:10.032520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:10.033120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:10.033391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:10.033655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:10.043249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:10.044671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:10.053755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1197/2392] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:22:15.485141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:15.485875 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:15.485962 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:15.522288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] MFA Rules not processed for user `5f4eeae86db248a09f0725ca23ac6a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:15.523701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:15.599253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1196/2393] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:15.604519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:15.616174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6251-3452-41b6-a9ac-65a8d64222a2 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:15.621909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:15.622564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:15.623026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:15.623393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:15.625483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:15.627537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:15.723144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1198/2394] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:22:15 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:22:15.728302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:15.739705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464f5dd-51f9-417f-8eab-4a79c1124d1e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:15.801660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:15.802356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:15.802637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:15.803078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:15.803690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:15.805745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:15.870016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1197/2395] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:22:15 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:22:15.873980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:15.884660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e43eb1-22c2-4e34-bca3-cf651964cc03 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:15.889320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:15.889908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:15.890202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:15.890452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:15.891127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:15.892710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:15.904846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1199/2396] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:22:15.909062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:15.920815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7617-fa7c-49e6-8c68-86b7871fd1bd admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:15.925538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:15.926183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:15.926503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:15.926815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] PATH_INFO: `/v3/projects/da97568ad65a4837ac139d200f430666/users/98185f0240bc42ebb0be13ddb155c040/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:15.943825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da97568ad65a4837ac139d200f430666, user_id=98185f0240bc42ebb0be13ddb155c040, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:15.947102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:16.001474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1198/2397] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:22:15 2022] PUT /identity/v3/projects/da97568ad65a4837ac139d200f430666/users/98185f0240bc42ebb0be13ddb155c040/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:22:16.005666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:16.017006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f61-5c9c-4261-ad75-6aae2675df59 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:16.074597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:16.075232 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:16.075506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:16.075772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:16.076412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:16.078035 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626710e4-7a88-4ac9-aea6-cb1bad95c679 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:16.089088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1200/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:22:16.093596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:16.104859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7ac94-62eb-4bac-983d-17b7d5523808 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:16.109588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:16.110307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:16.110938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:16.111289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] PATH_INFO: `/v3/projects/da97568ad65a4837ac139d200f430666/users/98185f0240bc42ebb0be13ddb155c040/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:16.113222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da97568ad65a4837ac139d200f430666, user_id=98185f0240bc42ebb0be13ddb155c040, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:16.116011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:16.128386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1199/2399] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:22:16 2022] PUT /identity/v3/projects/da97568ad65a4837ac139d200f430666/users/98185f0240bc42ebb0be13ddb155c040/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:22:21.324948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08f7e-31cc-4f3b-990b-43071936bd84 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:21.325468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08f7e-31cc-4f3b-990b-43071936bd84 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:21.325589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08f7e-31cc-4f3b-990b-43071936bd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:21.358109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-81c08f7e-31cc-4f3b-990b-43071936bd84 None None] MFA Rules not processed for user `98185f0240bc42ebb0be13ddb155c040`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:21.359536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-81c08f7e-31cc-4f3b-990b-43071936bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:21.393089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:21.440484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1201/2400] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:21.453951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:21.455969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:21.520306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:21.536571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46f287-3cd0-441f-bc23-4d9dfac0fb16 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:21.540361 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f814ce7767d645c58a5c00a9672c54eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:21.541062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:21.541413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:21.541751 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:21.612007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:21.613637 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:21.625802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1200/2401] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:22:21.908213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace631b2-c776-4222-80ce-2aec62b4e941 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:21.908497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace631b2-c776-4222-80ce-2aec62b4e941 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:21.908770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace631b2-c776-4222-80ce-2aec62b4e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:21.929539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ace631b2-c776-4222-80ce-2aec62b4e941 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:21.934905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ace631b2-c776-4222-80ce-2aec62b4e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:21.991595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1202/2402] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:22:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:22.004233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:22.005625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:22.081879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61a0e-0a4e-4fbb-be09-5ee3d35b231e service glance] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:22.088161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:22.088860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:22.089176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:22.089407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:22.098415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:22.100105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:22.109006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1201/2403] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:22:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:22:31.984406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=42 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:32.105557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=62 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:32.761816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0373c7-68c8-41ec-8ce4-05c31d908483 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:32.762101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0373c7-68c8-41ec-8ce4-05c31d908483 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:32.762374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0373c7-68c8-41ec-8ce4-05c31d908483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:32.781386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-0a0373c7-68c8-41ec-8ce4-05c31d908483 None None] MFA Rules not processed for user `5f4eeae86db248a09f0725ca23ac6a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:32.782623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0a0373c7-68c8-41ec-8ce4-05c31d908483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:32.790779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1203/2404] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:33.063895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:33.065393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:33.124496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa48c0-3db2-417e-8500-99bd6641ac77 service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:33.128298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:33.128904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:33.129183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:33.129546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:33.131216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:33.140995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:33.142421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f93216-8cb5-4be8-9d14-9df90e38c9ed service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:33.151367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1202/2405] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:22:34.339297 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:34.339812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:34.339977 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:34.358237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] MFA Rules not processed for user `18fae6990e4148ff97dec28b204c52df`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:34.361050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:34.416355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1204/2406] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:22:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:38.176091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:38.176708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:38.176708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:38.198015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:22:38.199296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:38.212778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1203/2407] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:22:38.230505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:38.243302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f682aa-d724-4200-9c9d-a5f9dbd50a24 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:38.249406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:38.251119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:38.251119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:38.251119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:38.260125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:38.261927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:38.270270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1205/2408] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:22:48.191921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=29 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:48.266973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=34 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:22:52.670774 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:52.672496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:52.758197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b3dc-734a-483a-ab34-d9140a493457 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:22:52.763925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:22:52.764530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:22:52.764805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:22:52.765075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:22:52.766895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:22:52.779342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:22:52.781076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58699a6c-b4c4-4e3e-8e66-ecc2fbcf748a service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:22:52.794065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1204/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:22:59.985033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:22:59.986806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:00.002952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e58be-f709-461b-a17f-6f7300a574ff service swift] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:23:00.015213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:23:00.016249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:00.016729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:00.017194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:00.019347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:00.092086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:23:00.094578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:23:00.105795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1206/2410] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:22:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:23:02.787670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=34 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:23:04.307610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:04.308216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:04.308429 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:04.328266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] MFA Rules not processed for user `528811040ede40bdb40e3e6e666d283d`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:23:04.329407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:04.388636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1205/2411] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:23:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:23:10.100687 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=28 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:23:14.148386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:23:14.159003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0b9ae-ae5c-4d5b-be67-932b5042d961 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:23:14.162528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:23:14.163158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:14.163433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:14.163697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:14.173944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:23:14.175393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26623-a5f5-4462-abbf-9149d4867d05 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:23:14.185861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1207/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:23:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:23:14.381284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=25 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:23:24.180473 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:23:40.016023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:23:40.017577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:40.154579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0413fe-457b-4765-b347-7328cb31d3e1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:23:40.159573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:23:40.160193 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:40.160467 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:40.160735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:40.162470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:40.173837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:23:40.175437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b5780-e93c-4154-b905-4df8656b5a29 service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:23:40.185010 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1206/2413] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:23:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:23:41.334813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc48d7-f167-4bd9-a05b-3a72985f85df None None] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.335281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc48d7-f167-4bd9-a05b-3a72985f85df None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.335377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc48d7-f167-4bd9-a05b-3a72985f85df None None] PATH_INFO: `/` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.336735 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1208/2414] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:23:41.343135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2f7cd-9177-4175-b642-067c9ab73aac None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.343643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2f7cd-9177-4175-b642-067c9ab73aac None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.343749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2f7cd-9177-4175-b642-067c9ab73aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.386524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-6ff2f7cd-9177-4175-b642-067c9ab73aac None None] MFA Rules not processed for user `a91c9f4d365a49d7b466292f5078fced`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:23:41.387886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6ff2f7cd-9177-4175-b642-067c9ab73aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:41.394546 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1207/2415] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:23:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:23:41.407130 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc48d7-f167-4bd9-a05b-3a72985f85df None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:23:41.420189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc48d7-f167-4bd9-a05b-3a72985f85df None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:23:41.423789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:23:41.424373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.424648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.424917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.431890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:23:41.433301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f9339-b0cd-4df6-8b20-42b4d2ef7b7a service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:23:41.441803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1209/2416] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:23:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:23:41.529035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39d1-097f-4012-8aa0-8d81ab7936a9 None None] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.529035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39d1-097f-4012-8aa0-8d81ab7936a9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.529035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39d1-097f-4012-8aa0-8d81ab7936a9 None None] PATH_INFO: `/` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.529035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1208/2417] 10.222.0.36 () {54 vars in 921 bytes} [Tue Aug 9 18:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:23:41.534268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.534557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.534894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.567227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] MFA Rules not processed for user `89d8460000674bcead45625785d81867`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:23:41.568521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:23:41.594328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1210/2418] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Aug 9 18:23:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:23:41.604874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada39d1-097f-4012-8aa0-8d81ab7936a9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:23:41.616484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada39d1-097f-4012-8aa0-8d81ab7936a9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:23:41.620024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:23:41.620603 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:23:41.620880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:23:41.621149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:23:41.631615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:23:41.633027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:23:41.643520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1209/2419] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:23:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:23:51.591301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=21 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:23:51.639768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=80 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:06.021348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:06.040126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03c168-af56-42a9-8399-82c1cd1ff223 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:06.043694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:06.044277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:06.044550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:06.044818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:06.046486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:06.128618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:06.130048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:06.149009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1211/2420] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:09.823729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:09.835098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4738-a269-4462-8c94-94db8760cc7c service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:09.840251 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:09.840849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:09.841126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:09.841421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:09.843246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:09.855169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:09.856758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:09.867743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1210/2421] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:09.954683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:09.956521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:09.970522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9963-7f65-4d6d-a23f-48cf5c187998 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:09.975742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:09.976372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:09.976649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:09.976946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] PATH_INFO: `/v3/users/18fae6990e4148ff97dec28b204c52df` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:09.978066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=18fae6990e4148ff97dec28b204c52df)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:09.980142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:10.130900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18fae6990e4148ff97dec28b204c52df'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.137586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18fae6990e4148ff97dec28b204c52df'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.149052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18fae6990e4148ff97dec28b204c52df'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.203544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18fae6990e4148ff97dec28b204c52df'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.204151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a5877fc7-2a1e-4c10-85d3-e9299545fbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18fae6990e4148ff97dec28b204c52df'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.210491 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1212/2422] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:09 2022] DELETE /identity/v3/users/18fae6990e4148ff97dec28b204c52df => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:10.441520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:10.452699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f3f6-cfad-4fb8-85e7-ae4563d24af8 service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:10.520567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:10.521187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:10.521461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:10.521729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] PATH_INFO: `/v3/projects/dbd7e50444cc4ef0873bfac9d9a2d514` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:10.522907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd7e50444cc4ef0873bfac9d9a2d514)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:10.524783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:10.668605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd7e50444cc4ef0873bfac9d9a2d514', 'request_id': 'req-a4df729b-6b7e-464b-bf3e-59113502738a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.674044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd7e50444cc4ef0873bfac9d9a2d514', 'request_id': 'req-a4df729b-6b7e-464b-bf3e-59113502738a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.680309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd7e50444cc4ef0873bfac9d9a2d514', 'request_id': 'req-a4df729b-6b7e-464b-bf3e-59113502738a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.764662 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] The token cache is being invalidate because project dbd7e50444cc4ef0873bfac9d9a2d514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:10.764951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.765559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a4df729b-6b7e-464b-bf3e-59113502738a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:10.767770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1211/2423] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:10 2022] DELETE /identity/v3/projects/dbd7e50444cc4ef0873bfac9d9a2d514 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:10.778587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:10.778974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:10.779274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:10.824751 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:10.827652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:10.828854 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:10.888364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1213/2424] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:10.898090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:10.898592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:10.898727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:10.918347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:10.919560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:10.928220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1212/2425] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:10.932443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:10.942931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d13598-fa97-410c-b567-f326a1adc1f8 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:10.947460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:10.948051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:10.948327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:10.948595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:10.949226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:24:10.949524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:10.951168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:10.963255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1214/2426] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:24:10.967478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:10.977071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77276f-f48e-42e8-8c8d-4dbedc570378 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:10.981642 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:10.982262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:10.982530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:10.982828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:10.983439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:10.985609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.061653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1213/2427] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:10 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:11.066586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.076468 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f149231-c8aa-4f7e-8a48-40dbb4a12f90 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.133167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.133758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.134032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.134294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.134958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.136888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.231981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1215/2428] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:11 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:11.237069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.247309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465f750-b0fc-4f03-9a28-766f4dde6517 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.252558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.253221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.253501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.253772 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.254422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.256029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.265634 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1214/2429] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:11.269516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.279419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4a8e-5feb-44eb-9532-15c3fd1dc914 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.284014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.284628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.284902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.285166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.285788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.287439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d4d6a-1724-4d2d-85f0-91e03126ebcc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.297764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1216/2430] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:11.302306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.312081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876daf0-99dd-47b8-8413-76724f8c588b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.317275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.317908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.318186 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.318454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] PATH_INFO: `/v3/projects/5415078c3c9846ea999209e6c450aff8/users/07e49a906f6b4647b9910fe176a3ae6e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.332750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5415078c3c9846ea999209e6c450aff8, user_id=07e49a906f6b4647b9910fe176a3ae6e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.335445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.429338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1215/2431] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:11 2022] PUT /identity/v3/projects/5415078c3c9846ea999209e6c450aff8/users/07e49a906f6b4647b9910fe176a3ae6e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:11.440301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433948-16da-4b42-b11b-30e48938180b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.440591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433948-16da-4b42-b11b-30e48938180b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.440863 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433948-16da-4b42-b11b-30e48938180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.469500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-28433948-16da-4b42-b11b-30e48938180b None None] MFA Rules not processed for user `07e49a906f6b4647b9910fe176a3ae6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:11.470745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-28433948-16da-4b42-b11b-30e48938180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:11.534637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1217/2432] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:11.539149 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.553667 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:11.601243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580a0d-7b3f-45e2-9ca4-aff11f2423f9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.606379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.606971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.607248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.607516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.608138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.610286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.713633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1216/2433] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:11 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:11.717760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433948-16da-4b42-b11b-30e48938180b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.724019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:11.729577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433948-16da-4b42-b11b-30e48938180b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.784846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.785835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.786278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.786962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.787595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.789553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.860658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1218/2434] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:11 2022] POST /identity/v3/users => generated 302 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:11.865202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.875418 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1d3be-1789-4aba-b86e-bea73db76254 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.880720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.881304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.881670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.881940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.882598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.884205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:11.895646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1217/2435] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:11.900011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:11.910472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047eb9-ce67-4cd2-bf5b-3cac907cc6af admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:11.915298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:11.915935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:11.916270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:11.916564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] PATH_INFO: `/v3/projects/40e7d6a458004f07bbf0c87fa3b65107/users/4b973e27324b4139a3164e60eb820d8f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:11.931600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e7d6a458004f07bbf0c87fa3b65107, user_id=4b973e27324b4139a3164e60eb820d8f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:11.934252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:12.035711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1219/2436] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:11 2022] PUT /identity/v3/projects/40e7d6a458004f07bbf0c87fa3b65107/users/4b973e27324b4139a3164e60eb820d8f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:12.039904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:12.050069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f7100-6f36-4231-abff-91ca6d147440 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:12.105128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:12.105731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:12.106025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:12.106279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:12.106939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:12.108924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc842be-87ee-49d9-86b4-c2b37e23627a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:12.119417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1218/2437] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:12.124450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:12.136108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023483-680c-4228-8614-411a77c9dd2b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:12.140672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:12.141310 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:12.141583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:12.141850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] PATH_INFO: `/v3/projects/40e7d6a458004f07bbf0c87fa3b65107/users/4b973e27324b4139a3164e60eb820d8f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:12.144966 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e7d6a458004f07bbf0c87fa3b65107, user_id=4b973e27324b4139a3164e60eb820d8f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:12.148143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:12.159564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1220/2438] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:12 2022] PUT /identity/v3/projects/40e7d6a458004f07bbf0c87fa3b65107/users/4b973e27324b4139a3164e60eb820d8f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:12.170089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:12.170403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:12.170684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:12.204379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] MFA Rules not processed for user `4b973e27324b4139a3164e60eb820d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:12.206288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:12.287053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1219/2439] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:12 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:12.299267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:12.300969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:12.394282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f50eb-6e1b-45f5-b9d9-e32c3bd86a6d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:12.398105 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:12.398716 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:12.399008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:12.399276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:12.408881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:12.410317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:12.420041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1221/2440] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:12.473132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:12.474550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:12.488706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c458a4-6fa6-44ae-9fc5-c11526777f8f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:12.492670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:12.493271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:12.493545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:12.493811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:12.499224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:12.563691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:12.566066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680fce4-2603-4940-97de-c38888c80535 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:12.577087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1220/2441] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:17.357857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:17.401235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:17.421573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8b5c-65dc-454d-b394-0e134a00cff7 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:17.426816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:17.427454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:17.427726 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:17.427992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] PATH_INFO: `/v3/users/07e49a906f6b4647b9910fe176a3ae6e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:17.429087 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e49a906f6b4647b9910fe176a3ae6e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:17.431032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:17.601577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e49a906f6b4647b9910fe176a3ae6e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:17.608199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e49a906f6b4647b9910fe176a3ae6e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:17.620144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e49a906f6b4647b9910fe176a3ae6e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:17.696940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e49a906f6b4647b9910fe176a3ae6e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:17.697559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e49a906f6b4647b9910fe176a3ae6e'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:17.703870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1222/2442] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:17 2022] DELETE /identity/v3/users/07e49a906f6b4647b9910fe176a3ae6e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:17.709121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:17.709616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:17.709728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:17.728205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:17.729380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:17.786238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1221/2443] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:17.796277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:17.797667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:17.865288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b9e2-338c-439d-af3c-de7126bcf545 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:17.868853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:17.869451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:17.869727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:17.869993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:17.879886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:17.881317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:17.890649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1223/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:18.336176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:18.336176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:18.345686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22177cd6-c00c-4c49-8bd0-eca62bd7ca5d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:18.350980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:18.351610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:18.351887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:18.352161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] PATH_INFO: `/v3/projects/5415078c3c9846ea999209e6c450aff8` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:18.353283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5415078c3c9846ea999209e6c450aff8)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:18.355214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:18.470132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5415078c3c9846ea999209e6c450aff8', 'request_id': 'req-836d51dc-b471-47e0-9422-c0bf139beae1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.476527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5415078c3c9846ea999209e6c450aff8', 'request_id': 'req-836d51dc-b471-47e0-9422-c0bf139beae1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.482911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5415078c3c9846ea999209e6c450aff8', 'request_id': 'req-836d51dc-b471-47e0-9422-c0bf139beae1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.553353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] The token cache is being invalidate because project 5415078c3c9846ea999209e6c450aff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:18.553641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.554283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.555873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1222/2445] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:18 2022] DELETE /identity/v3/projects/5415078c3c9846ea999209e6c450aff8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:18.560016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:18.570730 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5975b-5da2-43cd-aa27-ac7c6a859cdf service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:18.615843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:18.627332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:18.628454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:18.628879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:18.629151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] PATH_INFO: `/v3/users/4b973e27324b4139a3164e60eb820d8f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:18.630274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b973e27324b4139a3164e60eb820d8f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:18.632211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:18.893604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b973e27324b4139a3164e60eb820d8f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.900410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b973e27324b4139a3164e60eb820d8f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.912169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b973e27324b4139a3164e60eb820d8f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.976553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b973e27324b4139a3164e60eb820d8f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.977168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-855a9c13-6113-46ba-a4c7-b78c29ab6df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b973e27324b4139a3164e60eb820d8f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:18.983283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1224/2446] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:18 2022] DELETE /identity/v3/users/4b973e27324b4139a3164e60eb820d8f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:19.378269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:19.380323 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:19.393367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d51dc-b471-47e0-9422-c0bf139beae1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:19.423340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:19.452309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:19.452908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.453211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.453481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] PATH_INFO: `/v3/projects/40e7d6a458004f07bbf0c87fa3b65107` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.454601 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e7d6a458004f07bbf0c87fa3b65107)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:19.456732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:19.559714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e7d6a458004f07bbf0c87fa3b65107', 'request_id': 'req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:19.565139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e7d6a458004f07bbf0c87fa3b65107', 'request_id': 'req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:19.571846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e7d6a458004f07bbf0c87fa3b65107', 'request_id': 'req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:19.633698 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] The token cache is being invalidate because project 40e7d6a458004f07bbf0c87fa3b65107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:19.633988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:19.634606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-e4d1d261-f8dd-4d3e-bbf8-c8811fa45def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:19.636138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1223/2447] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:19 2022] DELETE /identity/v3/projects/40e7d6a458004f07bbf0c87fa3b65107 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:19.648282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.648801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.648801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.668380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:19.669572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:19.728741 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1225/2448] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:24:19.738472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.738910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.739235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.759581 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:19.760786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:19.768980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1224/2449] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:19.773318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:19.782930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bdb65-9885-4cfb-bacf-29feb1ed6930 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:19.788336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:19.788974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.789305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.789625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.790367 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:24:19.790764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:19.792482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:19.804494 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1226/2450] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:24:19.808685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:19.817952 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272b804-80db-4fe1-9f2c-f533de9a5605 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:19.822664 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:19.823280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.823556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.823886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.824508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:19.826392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:19.916629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1225/2451] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:19 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:19.920709 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:19.931408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed1da-6369-4666-a5eb-5f5f5c1c7aca admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:19.980132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:19.986945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:19.987533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:19.987805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:19.988765 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:19.989561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:19.991535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.120503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1227/2452] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:19 2022] POST /identity/v3/users => generated 304 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:20.124722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.135117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420eaa7-3285-4c4c-a608-1ec46f0321c4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.139618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.140211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.140490 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.140760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.141424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.143052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.152935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1226/2453] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:20.157295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.168039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077c066-cbd5-4f9a-bb23-b2357f3cacdf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.173579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.174167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.174447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.174745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.175383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.177635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093503f1-da33-490d-b06c-84c1a2cb9673 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.188360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1228/2454] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:20.192717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.202497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100f64a-9b36-4f3d-9997-80b5174ec4e1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.207596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.210825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.210825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.210825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] PATH_INFO: `/v3/projects/21cff9452bc8441ca412d33a0681c7be/users/9c8c06080a524ca8a3de62c7e97caff1/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.224168 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cff9452bc8441ca412d33a0681c7be, user_id=9c8c06080a524ca8a3de62c7e97caff1, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.226957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.291966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1227/2455] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:20 2022] PUT /identity/v3/projects/21cff9452bc8441ca412d33a0681c7be/users/9c8c06080a524ca8a3de62c7e97caff1/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:20.302590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.302924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.303199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.330392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] MFA Rules not processed for user `9c8c06080a524ca8a3de62c7e97caff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:20.331629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:20.395154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1229/2456] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:20 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:20.399922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.458538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:20.462838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d83d7-e7e5-445b-925d-5dc3965a754a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.467721 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.468331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.468708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.468858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.469503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.471443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.584899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1228/2457] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:20 2022] POST /identity/v3/projects => generated 409 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:20.588543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.599976 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ce2e8-9b77-4026-bff8-c9794958e751 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.656544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.657217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.657408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.657671 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.658322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.661013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.746606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1230/2458] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:20 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:20.750685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.761463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045decc-ad71-46c3-a661-a0c8bf086a0f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.765998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.766593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.766897 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.767175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.767834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.769745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.779276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1229/2459] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:20.783419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.794060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140def77-df4f-405d-8a3b-8b9f87a00835 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.798873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.799514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.799789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.800063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] PATH_INFO: `/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c/users/7044cc0dcc824eaeb1f9afbaefc2dbb9/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.816809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a011a5bb954c738eb1b86dfb21eb7c, user_id=7044cc0dcc824eaeb1f9afbaefc2dbb9, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.819471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:20.919618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1231/2460] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:20 2022] PUT /identity/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c/users/7044cc0dcc824eaeb1f9afbaefc2dbb9/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:20.923651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:20.934540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46374e-ff5d-4cb7-bbdd-e8c085ca1fe1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:20.990366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:20.990999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:20.991325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:20.991554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:20.992187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:20.993768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbea09-6c0d-4803-9641-8030856178e3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:21.003269 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1230/2461] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:21.007719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:21.017885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6570-3c66-4a96-86a9-f1e949130577 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:21.023151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:21.023784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:21.024060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:21.024344 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] PATH_INFO: `/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c/users/7044cc0dcc824eaeb1f9afbaefc2dbb9/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:21.026291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a011a5bb954c738eb1b86dfb21eb7c, user_id=7044cc0dcc824eaeb1f9afbaefc2dbb9, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:21.028942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:21.039548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1232/2462] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:21 2022] PUT /identity/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c/users/7044cc0dcc824eaeb1f9afbaefc2dbb9/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:21.050632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:21.050970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:21.051258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:21.079377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] MFA Rules not processed for user `7044cc0dcc824eaeb1f9afbaefc2dbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:21.080576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:21.149375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1231/2463] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:21.160764 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:21.162478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:21.180819 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:21.219861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4b52-45ef-4860-8646-177d70eabbec admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:21.224848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:21.225445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:21.225719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:21.225984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:21.235499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:21.236981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1964e1-bafd-4800-8df8-8bb5e97a8bcc service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:21.246583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1233/2464] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:21.608040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:21.618338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f90098-bad7-47b8-80bc-d37b1e3036b8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:21.621990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:21.625510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:21.625510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:21.625510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:21.686739 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:21.696838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:21.698295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:21.708625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1232/2465] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:21 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:31.250581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=19 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:32.024320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ee87-7bf6-4f8b-8e3e-1ef66e764550 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:32.024597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ee87-7bf6-4f8b-8e3e-1ef66e764550 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:32.024864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ee87-7bf6-4f8b-8e3e-1ef66e764550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:32.044375 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4d36ee87-7bf6-4f8b-8e3e-1ef66e764550 None None] MFA Rules not processed for user `a6c7401eaf744ef895bacc294f7c6268`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:32.045213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:32.045724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4d36ee87-7bf6-4f8b-8e3e-1ef66e764550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:32.101952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1234/2466] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:32.851322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:32.920176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d82dcd-b83e-40de-8e9b-9a75b164ce6c service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:32.925906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:32.926537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:32.927554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:32.927838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] PATH_INFO: `/v3/users/9c8c06080a524ca8a3de62c7e97caff1` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:32.929538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8c06080a524ca8a3de62c7e97caff1)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:32.932532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:33.143726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8c06080a524ca8a3de62c7e97caff1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:33.157782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8c06080a524ca8a3de62c7e97caff1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:33.158519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8c06080a524ca8a3de62c7e97caff1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:33.166735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8c06080a524ca8a3de62c7e97caff1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:33.167614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8c06080a524ca8a3de62c7e97caff1'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:33.241443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1233/2467] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:32 2022] DELETE /identity/v3/users/9c8c06080a524ca8a3de62c7e97caff1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:33.246163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:33.246163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:33.246876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:33.288659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:33.289961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:33.358733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1235/2468] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:33.370343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:33.372543 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:33.443618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c450ff-c330-4b21-8b00-24bd80b6203f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:33.447574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:33.448189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:33.448462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:33.448791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:33.461045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:33.461471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:33.474946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1234/2469] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:33.945198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:33.946624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:33.959372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006cece-fc75-4835-b388-0fa99e6eb929 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:33.964086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:33.964918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:33.965350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:33.965787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] PATH_INFO: `/v3/projects/21cff9452bc8441ca412d33a0681c7be` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:33.967191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cff9452bc8441ca412d33a0681c7be)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:33.969292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:34.103540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cff9452bc8441ca412d33a0681c7be', 'request_id': 'req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.109246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cff9452bc8441ca412d33a0681c7be', 'request_id': 'req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.115621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cff9452bc8441ca412d33a0681c7be', 'request_id': 'req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.209721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] The token cache is being invalidate because project 21cff9452bc8441ca412d33a0681c7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:34.210246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.210931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.212582 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1236/2470] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:33 2022] DELETE /identity/v3/projects/21cff9452bc8441ca412d33a0681c7be => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:34.216731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:34.227105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe0569-4abe-4733-a9cf-f155d420d938 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:34.246394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:34.291140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:34.291790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:34.292068 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:34.292344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] PATH_INFO: `/v3/users/7044cc0dcc824eaeb1f9afbaefc2dbb9` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:34.293497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7044cc0dcc824eaeb1f9afbaefc2dbb9)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:34.296759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:34.498622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7044cc0dcc824eaeb1f9afbaefc2dbb9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.511872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7044cc0dcc824eaeb1f9afbaefc2dbb9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.512586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7044cc0dcc824eaeb1f9afbaefc2dbb9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.519375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7044cc0dcc824eaeb1f9afbaefc2dbb9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.519989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-690eec60-0582-48ad-a1b9-3bf17ad6f9b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7044cc0dcc824eaeb1f9afbaefc2dbb9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:34.611000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1235/2471] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:34 2022] DELETE /identity/v3/users/7044cc0dcc824eaeb1f9afbaefc2dbb9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:35.052978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.054457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:35.067477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b956-20a2-4aa4-b35d-b8dbdb3f6928 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.097080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:35.131877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.132524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.132787 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.133054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] PATH_INFO: `/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.134200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a011a5bb954c738eb1b86dfb21eb7c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.136198 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.242847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a011a5bb954c738eb1b86dfb21eb7c', 'request_id': 'req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:35.248621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a011a5bb954c738eb1b86dfb21eb7c', 'request_id': 'req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:35.255810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a011a5bb954c738eb1b86dfb21eb7c', 'request_id': 'req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:35.332889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] The token cache is being invalidate because project d2a011a5bb954c738eb1b86dfb21eb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:35.335942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:35.335942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-f2e92bb9-b379-4365-98a0-e3fcd8dfbb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:35.336973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1237/2472] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:35 2022] DELETE /identity/v3/projects/d2a011a5bb954c738eb1b86dfb21eb7c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:35.342233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.342791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.343216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.366415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:35.371277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:35.434058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1236/2473] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:35.442003 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.442276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.442589 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.463089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:35.464293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:35.477442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1238/2474] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:35.481865 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.492763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3f42-c963-4b33-ab2c-f0e4e3e766f6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.497498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.498105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.498381 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.498646 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.499321 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:24:35.499618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.501204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.514938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1237/2475] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:24:35.518781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.528804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232a09d-70df-4217-90b0-5106e93aa285 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.533985 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.534593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.534951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.535220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.535842 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.538150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.639736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1239/2476] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:35 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:35.643929 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.654426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f4224-6ff8-403f-af4c-b770b59ba0d0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.686233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:35.721200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.721919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.722197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.722463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.723121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.725656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.867024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1238/2477] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:35 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:35.868332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.878539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757cd7-3a6c-4381-9796-7da500cac352 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.885036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.885644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.885917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.886184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.886857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.888462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.899378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1240/2478] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:35.903566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.914398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb410-516f-4291-9203-314e4309a628 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.919005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.919596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.931519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.931519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.931519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.931519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ecc71-34fd-4fab-931c-240caf68a98f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:35.932807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1239/2479] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:35.936934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:35.947249 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7f7a-760c-4008-a672-b6300ea2fdc9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:35.951879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:35.952519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:35.952781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:35.953049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] PATH_INFO: `/v3/projects/ae3818cc5ab841c3937f7c255d0bfd05/users/36de73ca0c04401b90090f79cd38bd9d/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:35.969442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3818cc5ab841c3937f7c255d0bfd05, user_id=36de73ca0c04401b90090f79cd38bd9d, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:35.972195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.045883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1241/2480] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:35 2022] PUT /identity/v3/projects/ae3818cc5ab841c3937f7c255d0bfd05/users/36de73ca0c04401b90090f79cd38bd9d/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:36.057317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.057618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.057923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.091786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] MFA Rules not processed for user `36de73ca0c04401b90090f79cd38bd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:36.093106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:36.169858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1240/2481] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:36.174383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.251713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:36.257245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73af30-725b-4638-9fa7-2571ff97fbb5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.262160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.262786 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.263065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.263335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.263959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.265986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.384484 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1242/2482] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:36 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:36.388565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.400515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31c49d-1f33-44b8-98e7-8b01fb4ce21c None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.478002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.478746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.479145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.479493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.480127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.482078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.563296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1241/2483] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:36 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:36.564690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.576194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31385dfe-6262-4e96-a378-9993f42a7617 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.581922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.582558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.582862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.583136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.583788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.585548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.595974 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1243/2484] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:36.600964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.611911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b533-4480-4420-8333-4eb97a801754 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.617714 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.618286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.618569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.618883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] PATH_INFO: `/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811/users/6e9daacd08064c1b8e099481cd1b3292/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.635574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b6260a3ab2484bab0b3c15e7d7e811, user_id=6e9daacd08064c1b8e099481cd1b3292, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.638407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.700860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1242/2485] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:36 2022] PUT /identity/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811/users/6e9daacd08064c1b8e099481cd1b3292/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:36.705158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.717453 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1c8-7eeb-4687-a5de-bd11142ddfac admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.773691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.774289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.774563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.774877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.775502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.777070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952d418-3898-4d73-9efa-45cafd71c734 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.787415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1244/2486] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:36.791566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.797006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:36.802377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb3be0-6269-4895-a400-0a9132f9c6e1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:36.807205 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:36.807842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.808119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.808385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] PATH_INFO: `/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811/users/6e9daacd08064c1b8e099481cd1b3292/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.810262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b6260a3ab2484bab0b3c15e7d7e811, user_id=6e9daacd08064c1b8e099481cd1b3292, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:36.812910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:36.823272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1243/2487] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:36 2022] PUT /identity/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811/users/6e9daacd08064c1b8e099481cd1b3292/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:36.834990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:36.835278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:36.835545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:36.864879 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] MFA Rules not processed for user `6e9daacd08064c1b8e099481cd1b3292`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:36.866123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:36.947964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1245/2488] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:36.958912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:36.961455 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:37.028794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a54ca-0786-40fb-b00f-dfc5bddea6d4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:37.033033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:37.033438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:37.033717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:37.033990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:37.045297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:37.047478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d01f4-9a50-45c5-935d-67bed3aad399 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:37.058171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1244/2489] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:38.340100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:38.390771 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:38.402760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8427c-c527-4a19-8ad1-45b89dd8b91c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:38.407303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:38.407922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:38.408196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:38.408477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] PATH_INFO: `/v3/users/36de73ca0c04401b90090f79cd38bd9d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:38.409575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36de73ca0c04401b90090f79cd38bd9d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:38.412102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:38.636360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36de73ca0c04401b90090f79cd38bd9d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:38.643191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36de73ca0c04401b90090f79cd38bd9d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:38.655624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36de73ca0c04401b90090f79cd38bd9d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:38.741792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36de73ca0c04401b90090f79cd38bd9d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:38.742507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36de73ca0c04401b90090f79cd38bd9d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:38.748899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1246/2490] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:38 2022] DELETE /identity/v3/users/36de73ca0c04401b90090f79cd38bd9d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:38.753536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:38.754041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:38.754153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:38.773609 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:38.774876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:38.833570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1245/2491] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:24:38.843664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:38.845067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:38.901488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854ecb-caad-42f7-8f21-e58461f96dc8 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:38.905027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:38.905623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:38.905895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:38.906158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:38.917393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:38.919134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:38.929158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1247/2492] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:39.376327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:39.377825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:39.390294 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e969cd6-2c0d-44c3-85a1-708f91128809 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:39.395049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:39.396350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:39.396631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:39.397125 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] PATH_INFO: `/v3/projects/ae3818cc5ab841c3937f7c255d0bfd05` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:39.398370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3818cc5ab841c3937f7c255d0bfd05)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:39.401121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:39.507958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3818cc5ab841c3937f7c255d0bfd05', 'request_id': 'req-8e008440-ac9c-4e85-9a83-abc8bf87b534'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.513419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3818cc5ab841c3937f7c255d0bfd05', 'request_id': 'req-8e008440-ac9c-4e85-9a83-abc8bf87b534'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.519544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3818cc5ab841c3937f7c255d0bfd05', 'request_id': 'req-8e008440-ac9c-4e85-9a83-abc8bf87b534'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.608300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] The token cache is being invalidate because project ae3818cc5ab841c3937f7c255d0bfd05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:39.608587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.609485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.611044 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1246/2493] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:39 2022] DELETE /identity/v3/projects/ae3818cc5ab841c3937f7c255d0bfd05 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:39.615750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:39.625295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f0bae-e14c-405c-95e7-aa473cf3f015 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:39.681289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:39.681902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:39.682176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:39.682444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] PATH_INFO: `/v3/users/6e9daacd08064c1b8e099481cd1b3292` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:39.683597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9daacd08064c1b8e099481cd1b3292)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:39.685518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:39.699597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:39.819902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9daacd08064c1b8e099481cd1b3292'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.827549 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9daacd08064c1b8e099481cd1b3292'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.839836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9daacd08064c1b8e099481cd1b3292'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.897012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9daacd08064c1b8e099481cd1b3292'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.897682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a2364484-1a4f-4899-a495-cea8544051d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9daacd08064c1b8e099481cd1b3292'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:39.904211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1248/2494] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:39 2022] DELETE /identity/v3/users/6e9daacd08064c1b8e099481cd1b3292 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:40.365776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:40.367300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:40.380556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e008440-ac9c-4e85-9a83-abc8bf87b534 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:40.393655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:40.443495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:40.444213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:40.444392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:40.444666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] PATH_INFO: `/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:40.445800 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b6260a3ab2484bab0b3c15e7d7e811)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:40.447868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:40.554109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b6260a3ab2484bab0b3c15e7d7e811', 'request_id': 'req-3b36bd35-1d76-41db-a022-5a2c86eb58ff'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:40.559526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b6260a3ab2484bab0b3c15e7d7e811', 'request_id': 'req-3b36bd35-1d76-41db-a022-5a2c86eb58ff'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:40.566685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b6260a3ab2484bab0b3c15e7d7e811', 'request_id': 'req-3b36bd35-1d76-41db-a022-5a2c86eb58ff'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:40.644655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] The token cache is being invalidate because project c2b6260a3ab2484bab0b3c15e7d7e811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:40.644867 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:40.646105 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-3b36bd35-1d76-41db-a022-5a2c86eb58ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:40.647628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1247/2495] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:40 2022] DELETE /identity/v3/projects/c2b6260a3ab2484bab0b3c15e7d7e811 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:40.652714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:40.653204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:40.653305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:40.673045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:40.674242 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:40.740652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1249/2496] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:40.748628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:40.748911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:40.749182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:40.769933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:40.772428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:40.780639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1248/2497] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:40.785032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:40.795722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6f7d6-714f-4981-959b-917b5b4557fd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:40.800457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:40.801054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:40.801330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:40.801663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:40.802314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:24:40.802611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:40.804354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:40.816544 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1250/2498] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:24:40.821510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:40.831180 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992361f2-7248-4058-b35a-e85cb9c56c41 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:40.837063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:40.837746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:40.838121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:40.838421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:40.839189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:40.841277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:40.940140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1249/2499] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:40 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:40.941304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:40.951864 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de60ea-c551-4bf4-a763-658c2833536f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.013025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.013634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.013913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.014184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.014845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.016802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.094777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1251/2500] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:40 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:41.098993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.109557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73468eb-0224-438c-bd01-60e911ae0b8e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.114930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.115572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.115842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.116104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.116732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.118331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.128037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1250/2501] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:41.133514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.143852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5123-9ba4-4be0-8ec3-b906c1feda48 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.148825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.149415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.149721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.149994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.150634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.152327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938edec-78c3-4add-ae13-cab0c7b57eee admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.154499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:41.163469 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1252/2502] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:41.167669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.178276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc2e9-c2ae-45a5-af8b-36a3f51f84b8 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.182888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.183520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.183791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.184054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] PATH_INFO: `/v3/projects/3e0650a219354360a3b86b4619cac9af/users/12cfb4204f1f409fbe666d583f15b4de/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.199345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0650a219354360a3b86b4619cac9af, user_id=12cfb4204f1f409fbe666d583f15b4de, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.202051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.267625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1251/2503] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:41 2022] PUT /identity/v3/projects/3e0650a219354360a3b86b4619cac9af/users/12cfb4204f1f409fbe666d583f15b4de/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:41.278177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.278463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.278767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.306696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] MFA Rules not processed for user `12cfb4204f1f409fbe666d583f15b4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:41.307919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:41.372141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1253/2504] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:24:41.376784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.428011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:41.446566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45dbd-d938-44e6-a302-32dc2e76c18e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.451498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.452101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.459669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.459669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.459669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.459669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.533397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1252/2505] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:41 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:41.539704 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.550404 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d0446-80ec-41d9-8a64-05abb88a77ff None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.607354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.607960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.608256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.608526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.609396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.611331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.702052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1254/2506] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:41 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:41.705977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.716409 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92782c-b7fa-4dfa-915f-9684a2dde9f3 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.720975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.721565 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.721843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.722113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.722888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.725161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.735117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1253/2507] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:41.739045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.749963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83237-0c5a-4e22-ba90-b7c4842499c2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.754533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.755262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.755532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.755801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.756428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.758006 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec10c45-ced9-431f-bc9c-0e5c7a15acaf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.767579 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1255/2508] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:41.771592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:41.781477 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b989d-9c04-4e2d-a4cb-c18895d15f39 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:41.786569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:41.787237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.787515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.787790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] PATH_INFO: `/v3/projects/981faf00c12142428da0a92f38ab546f/users/e77e74201c6d47bdb978adebb7501a22/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.802619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981faf00c12142428da0a92f38ab546f, user_id=e77e74201c6d47bdb978adebb7501a22, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:41.805363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:41.930734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1254/2509] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:41 2022] PUT /identity/v3/projects/981faf00c12142428da0a92f38ab546f/users/e77e74201c6d47bdb978adebb7501a22/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:41.941856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:41.942154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:41.942486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:41.970553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] MFA Rules not processed for user `e77e74201c6d47bdb978adebb7501a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:41.971722 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:42.025918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:42.034670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1256/2510] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:42.041338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.103358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35924-cc2c-4660-9bd1-b54041aa42f4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.109283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.109882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.110155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.110420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.111063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.113020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.191363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1255/2511] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:42 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:42.195533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.207493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65135e1c-e8d5-48bc-b96c-0fe9c4c19135 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.268137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.269028 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.269383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.269711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.271048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.273039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.371692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1257/2512] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:42 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:42.375976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.388713 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10220e-5e96-43d4-b064-e2967bf74e81 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.393592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.394514 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.394883 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.399471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.399471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.399471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.410001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1256/2513] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:42.414606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.425690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf97e-3742-4fc3-b4bc-07333ee2de89 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.431253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.432042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.432320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.432592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] PATH_INFO: `/v3/projects/da88df8074fc4f48b87937e96b3a2ae2/users/500dc1cb0ef141d29a949c63e69bdd5d/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.451376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da88df8074fc4f48b87937e96b3a2ae2, user_id=500dc1cb0ef141d29a949c63e69bdd5d, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.457077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.538467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1258/2514] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:42 2022] PUT /identity/v3/projects/da88df8074fc4f48b87937e96b3a2ae2/users/500dc1cb0ef141d29a949c63e69bdd5d/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:42.542319 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.553265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182051a-ed37-4dfd-b5f6-d961433a3e78 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.615597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.616478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.616769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.617031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.617665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.619282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321eb03-9a6b-4666-a03b-73772b1d973d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.630861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1257/2515] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:42.635132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.646783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b59a-c496-41a2-87e2-6f24d82a830a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.651536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.652183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.652461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.652729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] PATH_INFO: `/v3/projects/da88df8074fc4f48b87937e96b3a2ae2/users/500dc1cb0ef141d29a949c63e69bdd5d/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.654663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da88df8074fc4f48b87937e96b3a2ae2, user_id=500dc1cb0ef141d29a949c63e69bdd5d, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.657445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.669662 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1259/2516] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:42 2022] PUT /identity/v3/projects/da88df8074fc4f48b87937e96b3a2ae2/users/500dc1cb0ef141d29a949c63e69bdd5d/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:24:42.680692 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.680977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.681253 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.688842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:42.724444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] MFA Rules not processed for user `500dc1cb0ef141d29a949c63e69bdd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:42.725804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:42.801432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1258/2517] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:42.813701 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:42.815511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:42.873677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da75cb-9fc2-4c6f-b8fc-42aa8f95495a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:42.877289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:42.877876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:42.878147 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:42.878409 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:42.935070 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:42.936503 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:42.945781 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1260/2518] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:44.819273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:44.820700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:44.832366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dff899-6c47-45f6-82ef-4c7bcd1324a2 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:44.835989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:44.836561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:44.836834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:44.837099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:44.894336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:44.896474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:44.906434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1259/2519] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:46.447455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:46.451504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:46.456980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9a62d-57e0-4097-be37-ac8687407105 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:46.461873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:46.462633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:46.463172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:46.463450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:46.480274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:46.481734 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f429d96-2e8d-4d5e-954e-813c622084e8 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:46.493071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1261/2520] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:50.956638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:51.018755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4482e5-062a-498d-a584-a017bd72b0cd service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:51.024279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:51.024895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:51.025170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:51.025438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] PATH_INFO: `/v3/users/12cfb4204f1f409fbe666d583f15b4de` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:51.026556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12cfb4204f1f409fbe666d583f15b4de)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:51.028550 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:51.120794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:51.241249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12cfb4204f1f409fbe666d583f15b4de'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:51.252733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12cfb4204f1f409fbe666d583f15b4de'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:51.253398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12cfb4204f1f409fbe666d583f15b4de'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:51.261255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12cfb4204f1f409fbe666d583f15b4de'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:51.261872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12cfb4204f1f409fbe666d583f15b4de'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:51.334284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1260/2521] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:50 2022] DELETE /identity/v3/users/12cfb4204f1f409fbe666d583f15b4de => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:51.339958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:51.340443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:51.340542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:51.359606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:51.360782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:51.419633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1262/2522] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:51.429391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:51.430870 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:51.501336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a56b-d064-40da-9aa7-7676fc2d6424 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:51.505314 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:51.505927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:51.506191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:51.506980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:51.515942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:51.517365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:51.527665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1261/2523] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:51.978018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:51.981627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:51.994649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984393e4-8fe4-44f2-9770-df3aca4a6cff None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:51.999411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:52.000747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:52.001193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:52.001467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] PATH_INFO: `/v3/projects/3e0650a219354360a3b86b4619cac9af` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:52.002656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0650a219354360a3b86b4619cac9af)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:52.004598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:52.102806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0650a219354360a3b86b4619cac9af', 'request_id': 'req-e1b2ce45-d38a-444a-a586-3ce1853991ae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.108362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0650a219354360a3b86b4619cac9af', 'request_id': 'req-e1b2ce45-d38a-444a-a586-3ce1853991ae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.114817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0650a219354360a3b86b4619cac9af', 'request_id': 'req-e1b2ce45-d38a-444a-a586-3ce1853991ae'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.182693 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] The token cache is being invalidate because project 3e0650a219354360a3b86b4619cac9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:52.183004 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.183624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.185094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1263/2524] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:51 2022] DELETE /identity/v3/projects/3e0650a219354360a3b86b4619cac9af => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:52.189116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:52.199779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05948-f1af-4b6d-b9ba-e6d6691b9922 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:52.255521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:52.256134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:52.256496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:52.256790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] PATH_INFO: `/v3/users/e77e74201c6d47bdb978adebb7501a22` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:52.257882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77e74201c6d47bdb978adebb7501a22)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:52.259843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:52.624561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77e74201c6d47bdb978adebb7501a22'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.626428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:52.636707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77e74201c6d47bdb978adebb7501a22'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.639610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77e74201c6d47bdb978adebb7501a22'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.644415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77e74201c6d47bdb978adebb7501a22'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.645071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77e74201c6d47bdb978adebb7501a22'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:52.769135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1262/2525] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:52 2022] DELETE /identity/v3/users/e77e74201c6d47bdb978adebb7501a22 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:53.200049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:53.200921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:53.217505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2ce45-d38a-444a-a586-3ce1853991ae admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:53.282795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:53.283426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:53.283703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:53.283972 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] PATH_INFO: `/v3/projects/981faf00c12142428da0a92f38ab546f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:53.285086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=981faf00c12142428da0a92f38ab546f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:53.287050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:53.302939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:53.417746 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981faf00c12142428da0a92f38ab546f', 'request_id': 'req-cf6032ae-7add-4e59-8f5e-bf346908e03c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.423061 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981faf00c12142428da0a92f38ab546f', 'request_id': 'req-cf6032ae-7add-4e59-8f5e-bf346908e03c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.429374 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981faf00c12142428da0a92f38ab546f', 'request_id': 'req-cf6032ae-7add-4e59-8f5e-bf346908e03c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.507518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] The token cache is being invalidate because project 981faf00c12142428da0a92f38ab546f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:53.507799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.508422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.509905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1264/2526] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:53 2022] DELETE /identity/v3/projects/981faf00c12142428da0a92f38ab546f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:53.513936 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:53.528030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:53.528318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142945f3-e941-4351-91fc-364441a86363 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:53.585303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:53.585958 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:53.586232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:53.586496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] PATH_INFO: `/v3/users/500dc1cb0ef141d29a949c63e69bdd5d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:53.587625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500dc1cb0ef141d29a949c63e69bdd5d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:53.589521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:53.807568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500dc1cb0ef141d29a949c63e69bdd5d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.819600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500dc1cb0ef141d29a949c63e69bdd5d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.820268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500dc1cb0ef141d29a949c63e69bdd5d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.827449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500dc1cb0ef141d29a949c63e69bdd5d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.828081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a756c9f2-6aea-4c7a-9511-2dc27c622090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500dc1cb0ef141d29a949c63e69bdd5d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:53.901843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1263/2527] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:24:53 2022] DELETE /identity/v3/users/500dc1cb0ef141d29a949c63e69bdd5d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:54.315196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:54.316833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:54.330944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6032ae-7add-4e59-8f5e-bf346908e03c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:54.388217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:54.388837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.389107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.389370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] PATH_INFO: `/v3/projects/da88df8074fc4f48b87937e96b3a2ae2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.390449 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da88df8074fc4f48b87937e96b3a2ae2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:54.392424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:54.498181 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da88df8074fc4f48b87937e96b3a2ae2', 'request_id': 'req-33402f44-1ba6-4889-90bd-6a4099bdb954'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:54.503536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da88df8074fc4f48b87937e96b3a2ae2', 'request_id': 'req-33402f44-1ba6-4889-90bd-6a4099bdb954'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:54.510191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da88df8074fc4f48b87937e96b3a2ae2', 'request_id': 'req-33402f44-1ba6-4889-90bd-6a4099bdb954'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:54.586082 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] The token cache is being invalidate because project da88df8074fc4f48b87937e96b3a2ae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:24:54.586298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:54.586946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-33402f44-1ba6-4889-90bd-6a4099bdb954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:24:54.588523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1265/2528] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:24:54 2022] DELETE /identity/v3/projects/da88df8074fc4f48b87937e96b3a2ae2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:54.593666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.594152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.594316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.614562 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:54.617426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:54.657462 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:54.675176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1264/2529] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:54.741041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.741319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.741613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.761555 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:54.762862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:54.770926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1266/2530] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:54.775640 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:54.785483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661e2b8-ebdd-4129-91a6-2ac0cd67ebf0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:54.790773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:54.791362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.791643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.791910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.792545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:24:54.792843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:54.794431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:54.805752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1265/2531] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:24:54.810149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:54.821738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68004e7b-6b9a-4c18-b7e1-6cb7786e1a26 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:54.826414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:54.827025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.827304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.827573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.828184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:54.830079 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:54.907304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1267/2532] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:54 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:54.911558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:54.922656 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c637e1-25fc-47b5-848f-62d6191c9393 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:54.981021 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:54.981652 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:54.981931 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:54.982202 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:54.982887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:54.985074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.091384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1266/2533] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:54 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:55.095666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.102421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:55.106312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7877-2bf4-4d33-9ba7-b02774b51fde admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.111157 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.111839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.112193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.112493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.113121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.114699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.124828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1268/2534] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:55.129156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.139786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f3d9-18e5-4e49-a262-06ba58b3e6c1 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.144559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.145141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.145416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.145683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.146356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.148825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacbf2-77d7-405e-b3cd-d99f3d2cf3a7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.159363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1267/2535] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:55.164342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.174906 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdaea0-2628-4429-b26b-980b925073d2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.180871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.181519 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.181797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.182098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] PATH_INFO: `/v3/projects/8721292f7f9849958506f2a0702ac03f/users/3b6a661eb3964adda7de59b9bb380cf1/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.199123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8721292f7f9849958506f2a0702ac03f, user_id=3b6a661eb3964adda7de59b9bb380cf1, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.201773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.280443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1269/2536] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:55 2022] PUT /identity/v3/projects/8721292f7f9849958506f2a0702ac03f/users/3b6a661eb3964adda7de59b9bb380cf1/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:24:55.290377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.290724 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.291015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.320112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] MFA Rules not processed for user `3b6a661eb3964adda7de59b9bb380cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:55.321362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:55.387882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1268/2537] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:55.392605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.456436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e2367-9716-44bb-a2ab-60bd705462b5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.464388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.464993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.465269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.465944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.466154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.468099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.548857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1270/2538] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:55 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:55.553699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.566027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e75a8e-2c77-43ba-ae42-27e50eb743f8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.607825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:55.624729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.625338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.626419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.626718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.627641 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.629643 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.753306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1269/2539] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:55 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:24:55.757568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.768080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893b14-d6c0-4f9c-85fe-8defb5d04966 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.773160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.773784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.774063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.774335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.775121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.776778 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.788862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1271/2540] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:55.792838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.803138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e251fa-1b35-41e3-869b-28e14f4eb192 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.808860 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.809463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.809740 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.810009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.810644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.812402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acdde4-99ff-48fd-a227-c315c9a69f74 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.823791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1270/2541] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:55.830444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:55.840529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630b6a-a5c4-499f-959d-29804cf6bd4d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:55.845221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:55.845846 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.846119 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.846391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] PATH_INFO: `/v3/projects/486b731b75354384bebeafa51ebe26d1/users/81a5c2a0f4a84ecbbf52d6dbb4a902d3/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:55.861711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486b731b75354384bebeafa51ebe26d1, user_id=81a5c2a0f4a84ecbbf52d6dbb4a902d3, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:55.864461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:55.942292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1272/2542] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:55 2022] PUT /identity/v3/projects/486b731b75354384bebeafa51ebe26d1/users/81a5c2a0f4a84ecbbf52d6dbb4a902d3/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:55.952520 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:55.952820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:55.953079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.106003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] MFA Rules not processed for user `81a5c2a0f4a84ecbbf52d6dbb4a902d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:56.107143 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:56.170597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1271/2543] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:56.175097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.240540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bf006-0c40-4f5b-a538-b97c4dd9ae11 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.245126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.245719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.246002 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.246262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.246907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.248803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.331815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1273/2544] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:24:56 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:56.336769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.347407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deea6e-c2a3-4e87-8719-117b6414a25f None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.402487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.403026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.403303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.403575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.404187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.406083 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.484073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1272/2545] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:24:56 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:24:56.488086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.496086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:56.499916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb89c-697f-48ca-bf10-57afbe001a4e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.504612 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.505670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.505670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.505895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.506401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.508246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.519046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1274/2546] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:56.523790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.534093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55d718-e7ed-4a4c-80d8-e7966b380905 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.539558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.541956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.541956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.541956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] PATH_INFO: `/v3/projects/f9e9987dba3d485696243c1366e13af3/users/411b68f637fc4b15927689c26fc1001b/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.557650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e9987dba3d485696243c1366e13af3, user_id=411b68f637fc4b15927689c26fc1001b, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.560350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.662413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1273/2547] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:56 2022] PUT /identity/v3/projects/f9e9987dba3d485696243c1366e13af3/users/411b68f637fc4b15927689c26fc1001b/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:24:56.667220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.677439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8744-e629-4f59-beb2-0396ff8b37c4 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.733532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.734377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.734652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.734949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.735578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.737162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e4ad0-4bc8-4a09-986d-acbaf0495bc9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.747415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1275/2548] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:24:56.751568 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.761880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266458ea-a1c1-4b1b-a6d3-5e76f949f6d5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.766480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.767147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.767425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.767695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] PATH_INFO: `/v3/projects/f9e9987dba3d485696243c1366e13af3/users/411b68f637fc4b15927689c26fc1001b/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.769579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e9987dba3d485696243c1366e13af3, user_id=411b68f637fc4b15927689c26fc1001b, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:56.772327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:56.782406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1274/2549] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:24:56 2022] PUT /identity/v3/projects/f9e9987dba3d485696243c1366e13af3/users/411b68f637fc4b15927689c26fc1001b/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:24:56.794510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.794828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.795102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:56.823118 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] MFA Rules not processed for user `411b68f637fc4b15927689c26fc1001b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:24:56.824325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:56.894525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1276/2550] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:24:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:24:56.906265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:56.907997 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:56.914560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:24:56.965947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed082a56-ab1f-41e0-bad8-010d6e672379 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:56.969617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:56.970209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:56.970483 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:56.970777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:57.026956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:57.028385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:57.037141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1275/2551] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:24:58.931600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:24:58.933048 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:24:58.946067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e326d2-9403-4455-b111-1b53d599ae82 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:24:58.949695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:24:58.950292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:24:58.950568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:24:58.950872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:24:59.012621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:24:59.014067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:24:59.023665 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1277/2552] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:24:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:00.357370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:00.366082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c043-a3fd-4ea7-88d1-e7e7801b955b service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:00.370179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:00.370776 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:00.371053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:00.371318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:00.379935 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:00.381349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdba87c-f080-44f6-ac1b-0f86da51c73b service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:00.390859 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1276/2553] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:04.923158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:04.936922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:04.985967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234d7c5-f231-40a5-b121-4b884b27a939 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:04.990501 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:04.991161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:04.991439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:04.991710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] PATH_INFO: `/v3/users/3b6a661eb3964adda7de59b9bb380cf1` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:04.992825 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b6a661eb3964adda7de59b9bb380cf1)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:04.994770 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:05.189257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b6a661eb3964adda7de59b9bb380cf1'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.196121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b6a661eb3964adda7de59b9bb380cf1'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.207833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b6a661eb3964adda7de59b9bb380cf1'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.282867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b6a661eb3964adda7de59b9bb380cf1'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.283420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b6a661eb3964adda7de59b9bb380cf1'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.289565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1278/2554] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:25:04 2022] DELETE /identity/v3/users/3b6a661eb3964adda7de59b9bb380cf1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:05.294630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:05.295132 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:05.296779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:05.313982 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:05.315206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:05.373237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1277/2555] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:25:05.382947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:05.384346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:05.442156 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8d7-6a85-421e-8500-b7bf771e667b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:05.445979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:05.446610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:05.446907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:05.447175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:05.457098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:05.458536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:05.469703 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1279/2556] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:05.846693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:05.848282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:05.860518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931cb5a-57c0-46c1-b7e1-449506a00c66 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:05.865693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:05.866317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:05.866588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:05.866881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] PATH_INFO: `/v3/projects/8721292f7f9849958506f2a0702ac03f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:05.867974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8721292f7f9849958506f2a0702ac03f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:05.869890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:05.982647 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:05.989832 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8721292f7f9849958506f2a0702ac03f', 'request_id': 'req-c4b479c2-7a99-40cb-9201-a07ada157f7c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:05.995284 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8721292f7f9849958506f2a0702ac03f', 'request_id': 'req-c4b479c2-7a99-40cb-9201-a07ada157f7c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.002632 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8721292f7f9849958506f2a0702ac03f', 'request_id': 'req-c4b479c2-7a99-40cb-9201-a07ada157f7c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.095700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] The token cache is being invalidate because project 8721292f7f9849958506f2a0702ac03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:25:06.095919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.096554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.098082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1278/2557] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:25:05 2022] DELETE /identity/v3/projects/8721292f7f9849958506f2a0702ac03f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:06.102215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:06.112834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a969527-4983-4d4e-b129-b83238c28a02 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:06.145146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:06.170059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:06.170686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:06.171171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:06.171908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] PATH_INFO: `/v3/users/81a5c2a0f4a84ecbbf52d6dbb4a902d3` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:06.172946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a5c2a0f4a84ecbbf52d6dbb4a902d3)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:06.174903 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:06.310077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a5c2a0f4a84ecbbf52d6dbb4a902d3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.316914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a5c2a0f4a84ecbbf52d6dbb4a902d3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.330413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a5c2a0f4a84ecbbf52d6dbb4a902d3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.399271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a5c2a0f4a84ecbbf52d6dbb4a902d3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.399898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a5c2a0f4a84ecbbf52d6dbb4a902d3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:06.406570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1280/2558] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:25:06 2022] DELETE /identity/v3/users/81a5c2a0f4a84ecbbf52d6dbb4a902d3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:06.816472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:06.817965 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:06.832678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b479c2-7a99-40cb-9201-a07ada157f7c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:06.894478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:06.895053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:06.895326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:06.895594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] PATH_INFO: `/v3/projects/486b731b75354384bebeafa51ebe26d1` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:06.896707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486b731b75354384bebeafa51ebe26d1)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:06.898590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:07.000729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486b731b75354384bebeafa51ebe26d1', 'request_id': 'req-bfad15d9-85cc-451f-b101-a2aad8467b43'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.006289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486b731b75354384bebeafa51ebe26d1', 'request_id': 'req-bfad15d9-85cc-451f-b101-a2aad8467b43'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.012501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486b731b75354384bebeafa51ebe26d1', 'request_id': 'req-bfad15d9-85cc-451f-b101-a2aad8467b43'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.072072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] The token cache is being invalidate because project 486b731b75354384bebeafa51ebe26d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:25:07.072358 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.072968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.074530 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1279/2559] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:25:06 2022] DELETE /identity/v3/projects/486b731b75354384bebeafa51ebe26d1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:07.080042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:07.080747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:07.094247 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c268ee-cf79-4bbd-ab23-6cfa534d94e5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:07.151326 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:07.152013 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:07.152269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:07.152592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] PATH_INFO: `/v3/users/411b68f637fc4b15927689c26fc1001b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:07.154357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411b68f637fc4b15927689c26fc1001b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:07.156914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:07.327102 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411b68f637fc4b15927689c26fc1001b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.333806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411b68f637fc4b15927689c26fc1001b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.334925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:07.351490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411b68f637fc4b15927689c26fc1001b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.414176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411b68f637fc4b15927689c26fc1001b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.414816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6c367489-c99c-4fae-adfc-39e3ccf5ac02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411b68f637fc4b15927689c26fc1001b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:07.421252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1281/2560] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:25:07 2022] DELETE /identity/v3/users/411b68f637fc4b15927689c26fc1001b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:07.811764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:07.813280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:07.827117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad15d9-85cc-451f-b101-a2aad8467b43 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:07.879747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:07.885498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:07.886119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:07.886392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:07.886658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] PATH_INFO: `/v3/projects/f9e9987dba3d485696243c1366e13af3` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:07.888659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e9987dba3d485696243c1366e13af3)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:07.892277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.003255 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e9987dba3d485696243c1366e13af3', 'request_id': 'req-f5fb5596-f831-4280-b128-3cb62fd6025a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:08.008992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e9987dba3d485696243c1366e13af3', 'request_id': 'req-f5fb5596-f831-4280-b128-3cb62fd6025a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:08.015644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e9987dba3d485696243c1366e13af3', 'request_id': 'req-f5fb5596-f831-4280-b128-3cb62fd6025a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:08.093706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] The token cache is being invalidate because project f9e9987dba3d485696243c1366e13af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:25:08.093991 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:08.094611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f5fb5596-f831-4280-b128-3cb62fd6025a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:08.096161 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1280/2561] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:25:07 2022] DELETE /identity/v3/projects/f9e9987dba3d485696243c1366e13af3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:08.101095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.101380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.101654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.121452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:08.123366 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:08.181043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1282/2562] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:08.189902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.190196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.190469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.210593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:08.211811 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:08.219789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1281/2563] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:08.224145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.234439 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e025-e7a3-4ca0-b26b-a5765c10b93f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.239000 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.239586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.239862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.240126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.240759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:25:08.241056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.242638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.257947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1283/2564] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:25:08.259008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.268838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27d9cc-006a-4671-b1fb-f99448326057 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.273880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.274484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.274802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.276158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.276158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.277591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.360658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1282/2565] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:25:08 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:25:08.365678 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.376155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160e8d-260a-421b-9693-b76345ea9184 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.440862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.440862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.440862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.440862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.441296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.443289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.547303 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1284/2566] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:25:08 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:25:08.551395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.561737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da098-9b8f-4c29-85f3-2fe4dda959a2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.566379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.566998 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.567281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.567547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.568217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.569792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.581212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1283/2567] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:25:08.591394 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.601760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fb806-d3ce-4e67-afbc-012d596695e6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.606520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.607152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.610754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.610754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.610754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.610754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da69041-e197-40e4-a66e-ed8d956405c5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.621007 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1285/2568] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:25:08.625296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.635258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd0ea-57d9-4961-bf9d-2089804f1763 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.640737 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.641564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.641676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.641914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] PATH_INFO: `/v3/projects/eb3aa9fd0a124dfe822a66f424969beb/users/6013ed73af5b463b9a485a9a57e172b4/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.657134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3aa9fd0a124dfe822a66f424969beb, user_id=6013ed73af5b463b9a485a9a57e172b4, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.659801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.712540 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1284/2569] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:25:08 2022] PUT /identity/v3/projects/eb3aa9fd0a124dfe822a66f424969beb/users/6013ed73af5b463b9a485a9a57e172b4/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:08.729215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8808-e302-4e91-ad3b-191e21051092 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.729509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8808-e302-4e91-ad3b-191e21051092 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.729779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8808-e302-4e91-ad3b-191e21051092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.757927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ab9d8808-e302-4e91-ad3b-191e21051092 None None] MFA Rules not processed for user `6013ed73af5b463b9a485a9a57e172b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:08.759171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ab9d8808-e302-4e91-ad3b-191e21051092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:08.788369 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:08.824318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1286/2570] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:08.838134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:08.839845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:08.897175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176b1a-8512-43ff-8ca3-5cc5b1e4d8df admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:08.900803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:08.901402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:08.901678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:08.901949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:08.910921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:08.912338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aea02f-8d57-407b-8e50-3cee102c6ad5 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:08.921585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1285/2571] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:10.585527 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:10.587772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:10.587772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:10.609202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:10.610877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:10.610988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:10.610988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:10.610988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:10.632154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] MFA Rules not processed for user `6013ed73af5b463b9a485a9a57e172b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:10.633671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:10.641184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1286/2572] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:25:10.673171 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1287/2573] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:25:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:10.719204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:10.720639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:10.744031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:10.791847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3c85-b87f-43c7-8507-5771af5077a9 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:10.797405 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:10.798008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:10.802843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:10.802843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:10.806579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:10.808167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:10.817406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1287/2574] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:20.667872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=39 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:20.816045 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:20.817359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:20.888237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64d637-02a4-452e-99df-1ebdf89dcc2e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:20.893252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:20.893888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:20.894189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:20.894522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:20.896832 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:20.984054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:20.985647 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7480bf-b682-46c6-a813-531875227608 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:20.993421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=23 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:20.997215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1288/2575] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:23.362097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:23.371911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05568954-9e40-4771-ab49-02b314f8309a service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:23.376184 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:23.377047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:23.377541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:23.378012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:23.380029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:23.395605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:23.397043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:23.408738 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1288/2576] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:30.992025 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=54 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:31.209199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1c07-4699-49b5-b139-9a6daa214d0c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:31.209477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1c07-4699-49b5-b139-9a6daa214d0c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:31.209752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1c07-4699-49b5-b139-9a6daa214d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:31.229981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-95ab1c07-4699-49b5-b139-9a6daa214d0c None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:31.231215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-95ab1c07-4699-49b5-b139-9a6daa214d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:31.238672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1289/2577] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:25:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:31.251325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:31.262340 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e191a8b-1a55-441c-92de-a267146e6da4 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:31.268195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:31.268769 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:31.269057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:31.269356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:31.277188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:31.278771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:31.286937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1289/2578] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:32.225855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6846-eabb-4a9f-8b40-a00eade8dcb5 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:32.226324 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6846-eabb-4a9f-8b40-a00eade8dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:32.226427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6846-eabb-4a9f-8b40-a00eade8dcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:32.246505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-019e6846-eabb-4a9f-8b40-a00eade8dcb5 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:32.249305 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-019e6846-eabb-4a9f-8b40-a00eade8dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:32.255509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1290/2579] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:32.269120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:32.279636 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cf427-b6a5-442a-bada-b2d11c7b823d service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:32.285310 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:32.285889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:32.286165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:32.286431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:32.294443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:32.296079 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:32.304722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1290/2580] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:32.992918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4ac9-02fe-4ba6-8329-df68eea60f9f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:32.993202 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4ac9-02fe-4ba6-8329-df68eea60f9f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:32.993485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4ac9-02fe-4ba6-8329-df68eea60f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:33.012851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-82ac4ac9-02fe-4ba6-8329-df68eea60f9f None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:33.014168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-82ac4ac9-02fe-4ba6-8329-df68eea60f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:33.021512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1291/2581] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:33.045517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:33.045517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a0a0-c761-4ec0-97f1-f82ef0d7b0ec service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:33.050144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:33.050793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:33.051073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:33.051337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:33.059464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:33.061025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:33.069683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1291/2582] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:33.647633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092ea2-54f4-4fc0-bcf2-c6a0e54c2807 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:33.648112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092ea2-54f4-4fc0-bcf2-c6a0e54c2807 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:33.648210 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092ea2-54f4-4fc0-bcf2-c6a0e54c2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:33.668031 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-61092ea2-54f4-4fc0-bcf2-c6a0e54c2807 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:33.669254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-61092ea2-54f4-4fc0-bcf2-c6a0e54c2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:33.676975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1292/2583] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:25:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:33.689498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:33.701118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49256e60-1972-4e59-88f6-5145e47d6074 service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:33.707114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:33.707933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:33.708371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:33.709212 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:33.717957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:33.720693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:33.730871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1292/2584] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:34.038383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:34.038669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:34.038975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:34.060901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:34.062290 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:34.070350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1293/2585] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:25:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:34.083618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:34.093949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45580fb-1243-45cc-bec2-a706bb653e8c service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:34.099359 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:34.099946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:34.100218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:34.100485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:34.108741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:34.110650 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:34.120000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1293/2586] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:25:36.113010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:36.114546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:36.131038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dd6d1-bdfd-416b-bf8e-fbf2498693cf None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:36.137050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:36.137952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:36.138458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:36.138919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:36.141465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:36.229961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:36.232167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf9813-5817-41ea-8293-8299b9a0874b service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:36.255575 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1294/2587] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:36.442456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:36.513087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4a463-7520-453d-88b7-68b3dad8d958 service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:36.518529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:36.519276 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:36.519619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:36.519954 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] PATH_INFO: `/v3/users/6013ed73af5b463b9a485a9a57e172b4` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:36.521155 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6013ed73af5b463b9a485a9a57e172b4)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:36.523527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:36.721553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6013ed73af5b463b9a485a9a57e172b4'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:36.733281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6013ed73af5b463b9a485a9a57e172b4'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:36.734003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6013ed73af5b463b9a485a9a57e172b4'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:36.741607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6013ed73af5b463b9a485a9a57e172b4'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:36.742316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6013ed73af5b463b9a485a9a57e172b4'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:36.856416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:36.861487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1294/2588] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:25:36 2022] DELETE /identity/v3/users/6013ed73af5b463b9a485a9a57e172b4 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:36.868515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:36.868820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:36.870570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:36.889939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:36.891652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:36.957826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1295/2589] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:25:36.967592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:36.969704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:37.029362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e99db2-3a41-4632-9d94-655be87cfb06 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:37.033815 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:37.034417 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:37.034691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:37.035057 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:37.045362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:37.046893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:37.057368 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1295/2590] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:25:37.292076 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:37.294152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:37.306316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f716c-8965-45b5-83f9-4a8873848c44 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:37.311309 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:37.312054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:37.312337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:37.312606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] PATH_INFO: `/v3/projects/eb3aa9fd0a124dfe822a66f424969beb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:37.314387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3aa9fd0a124dfe822a66f424969beb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:37.316425 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:37.415885 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:37.428428 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3aa9fd0a124dfe822a66f424969beb', 'request_id': 'req-148e7a59-c1cf-4342-bc70-2ed487700382'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:37.434452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3aa9fd0a124dfe822a66f424969beb', 'request_id': 'req-148e7a59-c1cf-4342-bc70-2ed487700382'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:37.444570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3aa9fd0a124dfe822a66f424969beb', 'request_id': 'req-148e7a59-c1cf-4342-bc70-2ed487700382'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:37.511898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] The token cache is being invalidate because project eb3aa9fd0a124dfe822a66f424969beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:25:37.512506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:37.513206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:25:37.515131 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1296/2591] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:25:37 2022] DELETE /identity/v3/projects/eb3aa9fd0a124dfe822a66f424969beb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:25:39.003106 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:39.011969 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda104e7-e178-46d2-b80e-47d87dd6426c service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:39.064516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:39.065223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:39.065567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:39.065895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:39.067778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:39.129391 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:39.130979 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273a3f-edf8-45a3-9c0c-0d864668663d service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:39.140160 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1296/2592] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:47.533896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:49.136078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=74 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:25:55.429915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:55.431390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:55.444440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7a59-c1cf-4342-bc70-2ed487700382 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:55.448160 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:55.448788 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:55.449066 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:55.449341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:55.451113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:55.517429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:55.518914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:55.528550 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1297/2593] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:25:55.825960 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ed16-d660-4f2d-88b8-9efe97626b18 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:55.826503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ed16-d660-4f2d-88b8-9efe97626b18 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:55.826503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ed16-d660-4f2d-88b8-9efe97626b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:55.877684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8b72ed16-d660-4f2d-88b8-9efe97626b18 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:25:55.878910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8b72ed16-d660-4f2d-88b8-9efe97626b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:55.946850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1297/2594] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:25:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:25:55.961235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:25:55.961545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:25:56.058897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bbadf-1cf1-42a8-9fb2-d868cc494393 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:25:56.064933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:25:56.065535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:25:56.069506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:25:56.069506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:25:56.074552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:25:56.076255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:25:56.085168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1298/2595] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:25:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:26:00.959471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:00.959985 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:00.960127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:00.981157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:00.982385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:00.990887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1298/2596] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:26:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:01.004817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:01.016235 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa92559-d24a-4c65-a18a-7e0e384992f9 service swift] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:01.021755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:01.022377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:01.022658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:01.022959 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:01.033482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:01.035287 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:01.051385 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1299/2597] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:26:03.733559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:03.737018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:03.756760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fe746-0077-4921-a517-89b8e797fa66 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:03.761489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:03.762192 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:03.762475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:03.762781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] PATH_INFO: `/v3/users/46f8e315028640a7a2106c23018bf158` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:03.763953 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f8e315028640a7a2106c23018bf158)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:03.767145 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:03.903766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f8e315028640a7a2106c23018bf158'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:03.916923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f8e315028640a7a2106c23018bf158'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:03.917770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f8e315028640a7a2106c23018bf158'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:03.926033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f8e315028640a7a2106c23018bf158'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:03.926673 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f8e315028640a7a2106c23018bf158'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.017053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1299/2598] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:26:03 2022] DELETE /identity/v3/users/46f8e315028640a7a2106c23018bf158 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:04.310530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:04.325481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fed46-4fd0-4512-914e-07dbe0e1e5d3 service swift] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:04.388981 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:04.401137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:04.401773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:04.402051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:04.402322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] PATH_INFO: `/v3/projects/ab313819a4b141fc8391e750896f8b33` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:04.403536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab313819a4b141fc8391e750896f8b33)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:04.405454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:04.520274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab313819a4b141fc8391e750896f8b33', 'request_id': 'req-a6fef017-1aa2-42ad-9783-34f078742c65'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.526962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab313819a4b141fc8391e750896f8b33', 'request_id': 'req-a6fef017-1aa2-42ad-9783-34f078742c65'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.534099 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab313819a4b141fc8391e750896f8b33', 'request_id': 'req-a6fef017-1aa2-42ad-9783-34f078742c65'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.603101 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] The token cache is being invalidate because project ab313819a4b141fc8391e750896f8b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:26:04.603318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.603916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.605396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1300/2599] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:26:04 2022] DELETE /identity/v3/projects/ab313819a4b141fc8391e750896f8b33 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:04.610024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:04.611438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:04.624028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d97622-d2c9-48f7-805f-2c3214c9608a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:04.663575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:04.693987 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:04.694616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:04.694912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:04.695176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] PATH_INFO: `/v3/users/0bc4e960be8e4445b2ffe4254d337ae0` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:04.696342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc4e960be8e4445b2ffe4254d337ae0)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:04.698349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:04.909367 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc4e960be8e4445b2ffe4254d337ae0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.923978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc4e960be8e4445b2ffe4254d337ae0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.927273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc4e960be8e4445b2ffe4254d337ae0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.931838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc4e960be8e4445b2ffe4254d337ae0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:04.932602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc4e960be8e4445b2ffe4254d337ae0'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.038430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1300/2600] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:26:04 2022] DELETE /identity/v3/users/0bc4e960be8e4445b2ffe4254d337ae0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:05.316721 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:05.318267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:05.343955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef017-1aa2-42ad-9783-34f078742c65 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:05.512652 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:05.513391 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:05.513577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:05.516432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] PATH_INFO: `/v3/projects/67a364027eec4fa3b5d8fc39cda46592` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:05.518695 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a364027eec4fa3b5d8fc39cda46592)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:05.522090 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:05.671138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a364027eec4fa3b5d8fc39cda46592', 'request_id': 'req-cdc51943-6eb1-454f-9fa7-c859da22080f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.687126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a364027eec4fa3b5d8fc39cda46592', 'request_id': 'req-cdc51943-6eb1-454f-9fa7-c859da22080f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.694564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a364027eec4fa3b5d8fc39cda46592', 'request_id': 'req-cdc51943-6eb1-454f-9fa7-c859da22080f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.782862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] The token cache is being invalidate because project 67a364027eec4fa3b5d8fc39cda46592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:26:05.797559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.797559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:05.797559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1301/2601] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:26:05 2022] DELETE /identity/v3/projects/67a364027eec4fa3b5d8fc39cda46592 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:09.662233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:09.663928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:09.746153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1dc38-2df2-4361-a509-b9f004aa20bc admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:09.794089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:09.795986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:09.810236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:09.813490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:09.819932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:09.820538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:09.820810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:09.821075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:09.822940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:09.834376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:09.836114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:09.846026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1301/2602] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:09.869015 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51943-6eb1-454f-9fa7-c859da22080f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:09.873905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:09.875077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:09.875467 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:09.875739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] PATH_INFO: `/v3/users/5f4eeae86db248a09f0725ca23ac6a9f` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:09.877152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4eeae86db248a09f0725ca23ac6a9f)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:09.878934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:10.058152 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4eeae86db248a09f0725ca23ac6a9f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.065655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4eeae86db248a09f0725ca23ac6a9f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.076992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4eeae86db248a09f0725ca23ac6a9f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.117505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4eeae86db248a09f0725ca23ac6a9f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.118121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4eeae86db248a09f0725ca23ac6a9f'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.124194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1302/2603] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:26:09 2022] DELETE /identity/v3/users/5f4eeae86db248a09f0725ca23ac6a9f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:10.365750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:10.376466 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947cae6-f0f8-4857-8067-c94b7d2fe7f9 service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:10.434188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:10.434839 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:10.435116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:10.435385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] PATH_INFO: `/v3/projects/d793302181944f85ade371ded12ca4c4` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:10.436487 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d793302181944f85ade371ded12ca4c4)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:10.438635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:10.539878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd793302181944f85ade371ded12ca4c4', 'request_id': 'req-d6a00138-c6e9-42d1-b039-b335f8d1077a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.545317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd793302181944f85ade371ded12ca4c4', 'request_id': 'req-d6a00138-c6e9-42d1-b039-b335f8d1077a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.551785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd793302181944f85ade371ded12ca4c4', 'request_id': 'req-d6a00138-c6e9-42d1-b039-b335f8d1077a'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.637629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] The token cache is being invalidate because project d793302181944f85ade371ded12ca4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:26:10.637928 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.638580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:10.640633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1302/2604] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:26:10 2022] DELETE /identity/v3/projects/d793302181944f85ade371ded12ca4c4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:10.644829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:10.646298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:10.660059 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a6e0c-608c-4bd1-bd89-38b34fd2c919 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:10.722085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:10.722890 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:10.723177 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:10.723475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] PATH_INFO: `/v3/users/98185f0240bc42ebb0be13ddb155c040` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:10.724773 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98185f0240bc42ebb0be13ddb155c040)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:10.727154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:11.009496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:11.022834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98185f0240bc42ebb0be13ddb155c040'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.029471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98185f0240bc42ebb0be13ddb155c040'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.041812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98185f0240bc42ebb0be13ddb155c040'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.116433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98185f0240bc42ebb0be13ddb155c040'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.117033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98185f0240bc42ebb0be13ddb155c040'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.123110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1303/2605] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:26:10 2022] DELETE /identity/v3/users/98185f0240bc42ebb0be13ddb155c040 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:11.348175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:11.349734 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:11.363065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a00138-c6e9-42d1-b039-b335f8d1077a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:11.423158 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:11.423790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:11.424074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:11.424336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] PATH_INFO: `/v3/projects/da97568ad65a4837ac139d200f430666` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:11.425449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=da97568ad65a4837ac139d200f430666)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:11.427397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:11.450399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:11.546313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da97568ad65a4837ac139d200f430666', 'request_id': 'req-91b932b0-745e-4590-9973-2e524d584dec'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.552435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da97568ad65a4837ac139d200f430666', 'request_id': 'req-91b932b0-745e-4590-9973-2e524d584dec'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.558458 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da97568ad65a4837ac139d200f430666', 'request_id': 'req-91b932b0-745e-4590-9973-2e524d584dec'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.664266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] The token cache is being invalidate because project da97568ad65a4837ac139d200f430666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:26:11.664558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.665226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:11.666809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1303/2606] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:26:11 2022] DELETE /identity/v3/projects/da97568ad65a4837ac139d200f430666 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:21.180420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=17,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:21.792235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:32.774892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:32.778624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:32.865507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08aa5f-8c8f-4160-a74e-3007a10af511 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:32.946362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:32.947098 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:32.947396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:32.947648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:32.949916 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:32.965621 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:32.970293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:32.987568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1304/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:26:38.986416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:38.987754 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:39.042777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b932b0-745e-4590-9973-2e524d584dec admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:39.046183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:39.046798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:39.047073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:39.047337 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:39.049576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:39.058597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:39.060045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:39.068509 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1304/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:39.317360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:39.318873 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:39.330806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882b70c-ebb8-4bd6-bc1e-51134563cc44 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:39.334438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:39.335243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:39.335522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:39.335792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:39.337513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:39.403971 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:39.405422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:39.415634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1305/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:43.105905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:43.107761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:43.123749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c85979-2d28-47a9-8254-a00cc61b638f service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:43.129135 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:43.129750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:43.130030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:43.130303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] PATH_INFO: `/v3/users/a6c7401eaf744ef895bacc294f7c6268` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:43.131528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c7401eaf744ef895bacc294f7c6268)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:43.133442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:43.359437 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c7401eaf744ef895bacc294f7c6268'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.372341 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c7401eaf744ef895bacc294f7c6268'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.373048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c7401eaf744ef895bacc294f7c6268'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.380657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c7401eaf744ef895bacc294f7c6268'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.381309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56c308d0-d8e1-42dd-8a34-a3e7c824dda0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c7401eaf744ef895bacc294f7c6268'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.459566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1305/2610] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:26:43 2022] DELETE /identity/v3/users/a6c7401eaf744ef895bacc294f7c6268 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:43.679868 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:43.694970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992b59-9238-4f78-8b0b-698a793be219 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:43.711584 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:43.761154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:43.761792 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:43.762069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:43.762336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] PATH_INFO: `/v3/projects/b23fb97c7c4049c49f6bad8418fc4dec` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:43.763611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23fb97c7c4049c49f6bad8418fc4dec)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:43.765595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:43.993167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23fb97c7c4049c49f6bad8418fc4dec', 'request_id': 'req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:43.998729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23fb97c7c4049c49f6bad8418fc4dec', 'request_id': 'req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:44.005907 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23fb97c7c4049c49f6bad8418fc4dec', 'request_id': 'req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:44.083401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] The token cache is being invalidate because project b23fb97c7c4049c49f6bad8418fc4dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:26:44.083713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:44.084332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-dc7b049e-b1ed-4607-92bf-5b0cc04cba6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:26:44.085821 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1306/2611] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:26:43 2022] DELETE /identity/v3/projects/b23fb97c7c4049c49f6bad8418fc4dec => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:26:44.090697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.090999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.091268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.111881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:44.113071 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:44.170672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1306/2612] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:44.180669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.180954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.181227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.200838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:44.202016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:44.210122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1307/2613] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:44.214416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.224566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c032bb-64fe-49f3-8999-2ed7cecb6d89 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.229097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.229680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.229957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.230224 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.230920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:26:44.231220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.233056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.245745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1307/2614] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:26:44.250192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.260044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60b7b4-4cf6-489a-ac90-5c8b2b8d6ff0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.264670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.265248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.265518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.265779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.266392 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.268881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.357979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1308/2615] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:26:44 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:26:44.362955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.367978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:44.373176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99543-f0fa-4e41-b609-874d569b646d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.428980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.429577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.429862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.430146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.430781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.432695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.506037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1308/2616] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:26:44 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:26:44.510620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.520877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6df3-a3f0-486c-a221-c0c71d9d7fd5 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.525378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.525953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.526225 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.526485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.527193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.528744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.539128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1309/2617] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:26:44.542975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.553567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9032d-7a31-4524-adda-631610b5a935 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.558157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.558759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.559040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.559311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.559966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.561555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea9d7b-0b40-4d86-bd82-5c84593d77de admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.572085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1309/2618] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:26:44.580595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.590316 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030cefc-550c-4914-9390-d4bbca460b93 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.595436 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.596127 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.596400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.596664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] PATH_INFO: `/v3/projects/ef103f8a63404f918eaa5dd0c626b4c0/users/b5ffbaec36054aa3a89cca8b6007c8bb/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.611562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef103f8a63404f918eaa5dd0c626b4c0, user_id=b5ffbaec36054aa3a89cca8b6007c8bb, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.615020 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.728312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1310/2619] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:26:44 2022] PUT /identity/v3/projects/ef103f8a63404f918eaa5dd0c626b4c0/users/b5ffbaec36054aa3a89cca8b6007c8bb/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:26:44.733730 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1f29b-05f1-4d17-b208-6016685d9d47 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.734009 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1f29b-05f1-4d17-b208-6016685d9d47 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.734277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1f29b-05f1-4d17-b208-6016685d9d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.754502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2ae1f29b-05f1-4d17-b208-6016685d9d47 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:44.755723 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2ae1f29b-05f1-4d17-b208-6016685d9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:44.813347 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1310/2620] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:44.823226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:44.824912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:44.864802 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:44.881523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb057-f6b3-4cb9-9074-3fb2b4f43e58 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:44.885194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:44.885826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:44.886096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:44.886358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:44.896395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:44.897804 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:44.908358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1311/2621] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:50.175679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5716-ac81-4f6a-b355-a9978bb98401 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:50.176197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5716-ac81-4f6a-b355-a9978bb98401 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:50.176356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5716-ac81-4f6a-b355-a9978bb98401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:50.209807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-80bc5716-ac81-4f6a-b355-a9978bb98401 None None] MFA Rules not processed for user `b5ffbaec36054aa3a89cca8b6007c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:50.212289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-80bc5716-ac81-4f6a-b355-a9978bb98401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:50.286794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1311/2622] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:26:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:50.303223 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:50.304629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:50.377010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dedd1-21d7-42b9-b550-d24e6b552087 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:50.382384 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:50.383014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:50.383286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:50.383547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:50.391226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:50.393334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:50.402293 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1312/2623] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:52.885993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:52.886476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:52.886559 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:52.906819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] MFA Rules not processed for user `b5ffbaec36054aa3a89cca8b6007c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:26:52.908649 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:52.915919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1312/2624] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:26:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:26:53.246922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:53.248430 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:53.261214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67626ec-3bcb-4470-9006-69c5dea3e605 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:53.265077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:53.265673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:53.265952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:53.266218 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:53.268724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:53.278733 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:53.280166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:53.288692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1313/2625] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:26:53.712950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:26:53.714331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:53.724256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:26:53.792118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc150e-93ab-4652-b037-f50590051bf6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:26:53.797333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:26:53.797937 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:26:53.798222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:26:53.798491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:26:53.800717 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:26:53.812995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:26:53.815150 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:26:53.825099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1313/2626] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:26:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:03.285085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=50 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:03.449551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:03.458339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30639ad-7cc4-4510-8fcf-175f8e9a0029 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:03.462017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:03.462610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:03.462909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:03.463179 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:03.464898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:03.477365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:03.478817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:03.489440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1314/2627] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:03.820809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=30 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:07.278855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:07.279444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:07.280648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:07.282138 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:07.295784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b714eeb-b5da-4f83-a1f8-033487a984b3 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:07.299805 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381b857-b632-4619-b418-76fd4ee2daf6 service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:07.301039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:07.301840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:07.301937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:07.302155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] PATH_INFO: `/v3/users/528811040ede40bdb40e3e6e666d283d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:07.303261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=528811040ede40bdb40e3e6e666d283d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:07.305325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:07.305876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:07.306485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:07.306771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:07.307170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:07.308906 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:07.322385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:07.324749 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:07.337582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1314/2628] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:07.461753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528811040ede40bdb40e3e6e666d283d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:07.468619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528811040ede40bdb40e3e6e666d283d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:07.480601 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528811040ede40bdb40e3e6e666d283d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:07.545047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528811040ede40bdb40e3e6e666d283d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:07.545675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0bc9e20d-b3f1-4eeb-8683-3d16f09747ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528811040ede40bdb40e3e6e666d283d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:07.552497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1315/2629] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:07 2022] DELETE /identity/v3/users/528811040ede40bdb40e3e6e666d283d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:07.895921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:07.908118 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a7921-ffd1-4268-949b-4a4e355523bb service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:07.965013 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:07.965633 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:07.965905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:07.966188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] PATH_INFO: `/v3/projects/ac2e5c4bafff4b16bb9d82dacb5dd056` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:07.967306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2e5c4bafff4b16bb9d82dacb5dd056)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:07.969176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.099681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2e5c4bafff4b16bb9d82dacb5dd056', 'request_id': 'req-9d453c03-f4e7-4202-a602-b981ce41e489'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:08.107904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2e5c4bafff4b16bb9d82dacb5dd056', 'request_id': 'req-9d453c03-f4e7-4202-a602-b981ce41e489'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:08.115206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2e5c4bafff4b16bb9d82dacb5dd056', 'request_id': 'req-9d453c03-f4e7-4202-a602-b981ce41e489'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:08.191222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] The token cache is being invalidate because project ac2e5c4bafff4b16bb9d82dacb5dd056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:08.191605 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:08.192480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-9d453c03-f4e7-4202-a602-b981ce41e489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:08.194043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1315/2630] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:07 2022] DELETE /identity/v3/projects/ac2e5c4bafff4b16bb9d82dacb5dd056 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:08.199077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.199533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.199744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.219742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:08.221150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:08.283479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1316/2631] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:08.291500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.291959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.292058 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.312210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:08.313385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:08.322072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1316/2632] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:08.326420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.336166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95039ff5-9bf9-407b-aed9-5e3cae6a0c46 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.340605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.341180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.341450 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.341713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.342341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:08.342629 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.344226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.355957 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1317/2633] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:08.360335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.370091 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77529c7-c952-4df0-a33d-adf4ac5cec4b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.374682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.375353 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.375626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.375892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.376506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.379062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.484156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1317/2634] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:08 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:08.488992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.499038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a288f-17e7-4ad7-bedd-2f80b8c504bf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.541867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:08.555065 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.556009 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.556488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.556925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.557925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.560869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.665772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1318/2635] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:08 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:08.670108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.681011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3e7c0-9dc2-4880-a011-74cf367b37cb admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.685703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.686286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.686556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.686950 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.687625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.689188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.699433 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1318/2636] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:08.703500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.713635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb8cee-7fbb-43c8-adc4-e89298c43d49 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.719758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.719758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.719758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.720074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.720504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.722727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a17d-73a8-414a-a3f8-0b27a7857aaf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.733150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1319/2637] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:08.737731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.747566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f640-9a32-4d49-8662-ecb913d637aa admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:08.752384 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:08.753043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.753313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.753577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] PATH_INFO: `/v3/projects/6e3d1b91307d4d95b712c8bdefdef4e1/users/9c07e41bc9c44aef838df1bb1e618403/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.768254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3d1b91307d4d95b712c8bdefdef4e1, user_id=9c07e41bc9c44aef838df1bb1e618403, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:08.771209 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:08.864996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1319/2638] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:08 2022] PUT /identity/v3/projects/6e3d1b91307d4d95b712c8bdefdef4e1/users/9c07e41bc9c44aef838df1bb1e618403/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:08.870142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741641-08f7-4d35-b58b-ecf19840109c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:08.874535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741641-08f7-4d35-b58b-ecf19840109c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:08.874535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741641-08f7-4d35-b58b-ecf19840109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:08.890432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ba741641-08f7-4d35-b58b-ecf19840109c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:08.891649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ba741641-08f7-4d35-b58b-ecf19840109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:08.948852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1320/2639] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:08.959416 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:08.961072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:08.976275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:09.019442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523e0d2-2b49-429b-a4f5-e78aad6189fe admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:09.023110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:09.023766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:09.024039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:09.024306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:09.034270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:09.039763 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:09.045039 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1320/2640] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:10.324280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.324882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.324882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.353561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] MFA Rules not processed for user `9c07e41bc9c44aef838df1bb1e618403`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:10.354801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:10.418798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1321/2641] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:10.423407 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:10.433956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705e851-197e-4317-93f7-d1141d95f701 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:10.438607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:10.439383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.439794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.440185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.440878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:10.442866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:10.519171 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1321/2642] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:10 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:27:10.523193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:10.534343 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb805b-0e3c-4dd2-8310-4110da866cee None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:10.584460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:10.591019 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:10.591624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.591899 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.592169 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.592793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:10.594723 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:10.730745 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1322/2643] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:10 2022] POST /identity/v3/users => generated 308 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:10.733094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:10.743579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4661799-9387-4074-ab96-e79bdf0449ad admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:10.748828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:10.749428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.749708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.750011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.750657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:10.752780 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:10.764430 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1322/2644] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:10.771460 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:10.782153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57530099-b994-4ea2-89ac-6602889b23f1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:10.788330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:10.788993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.789270 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.789536 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] PATH_INFO: `/v3/projects/8b44a4d422714798a90a494a0756a543/users/b969956f647848da9dc2e4622a2d275c/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.804513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b44a4d422714798a90a494a0756a543, user_id=b969956f647848da9dc2e4622a2d275c, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:10.807199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:10.905691 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1323/2645] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:10 2022] PUT /identity/v3/projects/8b44a4d422714798a90a494a0756a543/users/b969956f647848da9dc2e4622a2d275c/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:27:10.909914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:10.919746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9fe12-d407-4c18-885c-06d278113605 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:10.976956 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:10.977453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:10.977731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:10.978003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:10.978628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:10.980241 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353594d-c9e1-4797-9c64-1324fd212bf5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:10.990732 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1323/2646] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:10.994951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:11.006686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d44e0-b133-44cd-80f9-d99d106478a1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:11.011685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:11.012445 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:11.012742 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:11.013011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] PATH_INFO: `/v3/projects/8b44a4d422714798a90a494a0756a543/users/b969956f647848da9dc2e4622a2d275c/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:11.014997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b44a4d422714798a90a494a0756a543, user_id=b969956f647848da9dc2e4622a2d275c, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:11.018201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:11.029043 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1324/2647] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:10 2022] PUT /identity/v3/projects/8b44a4d422714798a90a494a0756a543/users/b969956f647848da9dc2e4622a2d275c/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:12.339624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:12.340136 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:12.340244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:12.372484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] MFA Rules not processed for user `b969956f647848da9dc2e4622a2d275c`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:12.374228 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:12.459686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1324/2648] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:13.813719 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:13.876278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15805856-81c0-4f14-8a2d-25e7084df089 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:13.880923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:13.881537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:13.881814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:13.882083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] PATH_INFO: `/v3/users/9c07e41bc9c44aef838df1bb1e618403` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:13.883256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c07e41bc9c44aef838df1bb1e618403)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:13.885174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:14.072414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c07e41bc9c44aef838df1bb1e618403'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.079074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c07e41bc9c44aef838df1bb1e618403'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.091438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c07e41bc9c44aef838df1bb1e618403'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.156331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c07e41bc9c44aef838df1bb1e618403'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.156932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c07e41bc9c44aef838df1bb1e618403'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.163934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1325/2649] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:13 2022] DELETE /identity/v3/users/9c07e41bc9c44aef838df1bb1e618403 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:14.444666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:14.452070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:14.462006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6eeb69-96e9-4ab0-8f7d-f6fc304b132b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:14.524545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:14.525172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:14.525444 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:14.525707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] PATH_INFO: `/v3/projects/6e3d1b91307d4d95b712c8bdefdef4e1` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:14.526848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3d1b91307d4d95b712c8bdefdef4e1)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:14.528878 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:14.650864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3d1b91307d4d95b712c8bdefdef4e1', 'request_id': 'req-ed3d58a0-4734-4430-be84-4f0be6da1a2d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.656300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3d1b91307d4d95b712c8bdefdef4e1', 'request_id': 'req-ed3d58a0-4734-4430-be84-4f0be6da1a2d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.663533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3d1b91307d4d95b712c8bdefdef4e1', 'request_id': 'req-ed3d58a0-4734-4430-be84-4f0be6da1a2d'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.728298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] The token cache is being invalidate because project 6e3d1b91307d4d95b712c8bdefdef4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:14.728521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.729124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:14.730726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1325/2650] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:14 2022] DELETE /identity/v3/projects/6e3d1b91307d4d95b712c8bdefdef4e1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:14.735333 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:14.737514 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:14.751979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290f4c1-5f14-4947-beef-cfeae3c2e7f4 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:14.778798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:14.813133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:14.813762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:14.814348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:14.814348 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] PATH_INFO: `/v3/users/b969956f647848da9dc2e4622a2d275c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:14.815447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b969956f647848da9dc2e4622a2d275c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:14.817415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:15.034386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b969956f647848da9dc2e4622a2d275c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.041947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b969956f647848da9dc2e4622a2d275c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.054433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b969956f647848da9dc2e4622a2d275c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.127904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b969956f647848da9dc2e4622a2d275c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.131438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-b42f0cf6-310b-4545-be2a-4b98e35716ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b969956f647848da9dc2e4622a2d275c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.135472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1326/2651] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:14 2022] DELETE /identity/v3/users/b969956f647848da9dc2e4622a2d275c => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:15.410153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:15.412567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:15.427032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d58a0-4734-4430-be84-4f0be6da1a2d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:15.485195 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:15.485872 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:15.486220 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:15.486504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] PATH_INFO: `/v3/projects/8b44a4d422714798a90a494a0756a543` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:15.487613 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b44a4d422714798a90a494a0756a543)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:15.489517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:15.589869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:15.604342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b44a4d422714798a90a494a0756a543', 'request_id': 'req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.610029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b44a4d422714798a90a494a0756a543', 'request_id': 'req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.617181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b44a4d422714798a90a494a0756a543', 'request_id': 'req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.715781 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] The token cache is being invalidate because project 8b44a4d422714798a90a494a0756a543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:15.715981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.716596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b7665afa-8ece-4deb-9b5f-fe21de6cd7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:15.718082 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1326/2652] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:15 2022] DELETE /identity/v3/projects/8b44a4d422714798a90a494a0756a543 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:15.723358 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:15.723643 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:15.723910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:15.744072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:15.745243 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:15.804910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1327/2653] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:15.812593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:15.812877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:15.813181 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:15.833051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:15.834240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:15.842003 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1327/2654] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:15.846256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:15.856393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb7980-6d61-45be-b598-d136d5d6fbfd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:15.861681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:15.862332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:15.862749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:15.863014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:15.863998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:15.864296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:15.865862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:15.877969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1328/2655] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:15.882283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:15.891421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9391d-4f16-4732-9f4e-9a2e482b9dd5 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:15.896449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:15.897031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:15.897304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:15.897566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:15.898179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:15.900107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.002588 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1328/2656] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:15 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:16.006859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.016939 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776da1a-9d4d-42e1-9bd3-618416f4d276 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.046335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:16.072801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.073388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.073658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.073923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.074559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.076768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.195395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1329/2657] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:16 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:16.199834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.210607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898fade-bffd-45c7-9f1e-822b4f11754e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.215221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.215804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.216076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.216338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.217085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.218553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.229060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1329/2658] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:16.233377 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.243998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d982baf-04f2-4f98-9857-9a99ef5bc965 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.248638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.249220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.249497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.249767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.250396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.252060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18566d30-0247-45c9-87f0-ddae793f6497 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.262500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1330/2659] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:27:16.266476 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.276338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58707d27-16ea-474f-81cb-fcbf33d1e29f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.280900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.281533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.281813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.282089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] PATH_INFO: `/v3/projects/43c047141003493392b7b52670755ff7/users/9f10d822513b4084a959d8a589dbfae9/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.297111 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c047141003493392b7b52670755ff7, user_id=9f10d822513b4084a959d8a589dbfae9, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.300352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.378334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1330/2660] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:16 2022] PUT /identity/v3/projects/43c047141003493392b7b52670755ff7/users/9f10d822513b4084a959d8a589dbfae9/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:16.390069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.390357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.390633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.418644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] MFA Rules not processed for user `9f10d822513b4084a959d8a589dbfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:16.419937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:16.484281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1331/2661] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:16.488838 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.549783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6be54-569b-4f34-9866-905c88c8fbbb admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.554247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.554864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.555137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.555401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.556006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.557876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.729313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1331/2662] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:16 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:27:16.733834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.744475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34989a6-7283-48f6-8ac9-ba1ccbe13edf None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.799748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.800458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.800683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.800951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.801599 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.803560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.876477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1332/2663] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:16 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:16.881199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.891785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1105d-ac0b-4db7-8c88-8cdba61de81d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.897557 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.898826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.898826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.898826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.899461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.901028 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:16.912611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1332/2664] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:16.917200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:16.927646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81c0a-ff1c-4b2b-8a77-bff870e4af88 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:16.932258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:16.932914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:16.933189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:16.933454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] PATH_INFO: `/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7/users/01839aa750b94d42aa5cc47051042b7f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:16.949245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0c0c54ba4c4bed9b05212d76ebc4b7, user_id=01839aa750b94d42aa5cc47051042b7f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:16.952610 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:17.013301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1333/2665] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:16 2022] PUT /identity/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7/users/01839aa750b94d42aa5cc47051042b7f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:17.018166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:17.028288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066249-b3ac-4b05-bc62-6bf1a50d6e3a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:17.060113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:17.085569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:17.086762 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:17.087045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:17.087316 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:17.087946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:17.089622 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae613d-fa94-4e05-a0ab-a7c59682d6c2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:17.100204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1333/2666] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:17.104495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:17.115872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772d3e4-6ee6-4fc2-b086-53c07117a45e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:17.120521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:17.121153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:17.121426 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:17.121697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] PATH_INFO: `/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7/users/01839aa750b94d42aa5cc47051042b7f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:17.123620 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0c0c54ba4c4bed9b05212d76ebc4b7, user_id=01839aa750b94d42aa5cc47051042b7f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:17.126227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:17.129089 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:17.137465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1334/2667] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:17 2022] PUT /identity/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7/users/01839aa750b94d42aa5cc47051042b7f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:17.148511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:17.148798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:17.149073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:17.178816 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] MFA Rules not processed for user `01839aa750b94d42aa5cc47051042b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:17.180012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:17.247089 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1334/2668] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:17.258670 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:17.260364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:17.316927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122fb3-b57c-4cf7-8350-bad783289757 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:17.320543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:17.321136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:17.321412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:17.321682 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:17.331096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:17.332516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015001b-a240-4099-ab24-229e8384141e service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:17.341704 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1335/2669] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:18.288128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:18.349266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57d4a-01f4-42ea-93ae-09b659897111 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:18.353964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:18.354583 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:18.354902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:18.355175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] PATH_INFO: `/v3/users/9f10d822513b4084a959d8a589dbfae9` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:18.356538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f10d822513b4084a959d8a589dbfae9)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:18.358459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:18.512988 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f10d822513b4084a959d8a589dbfae9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:18.521374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:18.525708 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f10d822513b4084a959d8a589dbfae9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:18.526366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f10d822513b4084a959d8a589dbfae9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:18.533175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f10d822513b4084a959d8a589dbfae9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:18.533799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f10d822513b4084a959d8a589dbfae9'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:18.596027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1335/2670] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:18 2022] DELETE /identity/v3/users/9f10d822513b4084a959d8a589dbfae9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:18.601706 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812097-be40-4269-ad94-70b21467558f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:18.602227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812097-be40-4269-ad94-70b21467558f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:18.602338 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812097-be40-4269-ad94-70b21467558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:18.621136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-10812097-be40-4269-ad94-70b21467558f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:18.622335 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-10812097-be40-4269-ad94-70b21467558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:18.679859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1336/2671] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:18.689597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:18.691333 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:18.748495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda55542-ee18-4b62-bbdc-d7de80b7c773 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:18.752238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:18.752895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:18.753177 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:18.753445 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:18.762420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:18.763891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:18.773798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1336/2672] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:19.195925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812097-be40-4269-ad94-70b21467558f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:19.197323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-10812097-be40-4269-ad94-70b21467558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:19.210042 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812097-be40-4269-ad94-70b21467558f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:19.214950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:19.219642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:19.219642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:19.219642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] PATH_INFO: `/v3/projects/43c047141003493392b7b52670755ff7` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:19.219642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c047141003493392b7b52670755ff7)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:19.219642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:19.311520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c047141003493392b7b52670755ff7', 'request_id': 'req-0e8b2a21-edd7-465e-a574-93c11f522cc0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.317489 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c047141003493392b7b52670755ff7', 'request_id': 'req-0e8b2a21-edd7-465e-a574-93c11f522cc0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.323547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c047141003493392b7b52670755ff7', 'request_id': 'req-0e8b2a21-edd7-465e-a574-93c11f522cc0'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.378982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] The token cache is being invalidate because project 43c047141003493392b7b52670755ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:19.379261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.379877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.381334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1337/2673] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:19 2022] DELETE /identity/v3/projects/43c047141003493392b7b52670755ff7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:19.385349 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:19.394864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f96e-8f42-42c6-a564-1c9f99444693 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:19.450804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:19.451422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:19.451695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:19.451959 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] PATH_INFO: `/v3/users/01839aa750b94d42aa5cc47051042b7f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:19.453052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01839aa750b94d42aa5cc47051042b7f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:19.455015 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:19.616299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01839aa750b94d42aa5cc47051042b7f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.628912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01839aa750b94d42aa5cc47051042b7f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.629619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01839aa750b94d42aa5cc47051042b7f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.636561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01839aa750b94d42aa5cc47051042b7f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.637996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5792fce6-88dd-4c9c-84b7-3c8f70740f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01839aa750b94d42aa5cc47051042b7f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:19.694064 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:19.699102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1337/2674] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:19 2022] DELETE /identity/v3/users/01839aa750b94d42aa5cc47051042b7f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:20.123635 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.126917 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:20.138044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2a21-edd7-465e-a574-93c11f522cc0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.155207 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:20.196935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.197548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.197814 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.198074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] PATH_INFO: `/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.199241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0c0c54ba4c4bed9b05212d76ebc4b7)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.201213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.281291 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0c0c54ba4c4bed9b05212d76ebc4b7', 'request_id': 'req-ce148fe3-05bd-4649-8c16-cb10be764dfe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:20.287875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0c0c54ba4c4bed9b05212d76ebc4b7', 'request_id': 'req-ce148fe3-05bd-4649-8c16-cb10be764dfe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:20.293850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0c0c54ba4c4bed9b05212d76ebc4b7', 'request_id': 'req-ce148fe3-05bd-4649-8c16-cb10be764dfe'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:20.401960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] The token cache is being invalidate because project 5b0c0c54ba4c4bed9b05212d76ebc4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:20.402373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:20.403184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-ce148fe3-05bd-4649-8c16-cb10be764dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:20.404803 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1338/2675] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:20 2022] DELETE /identity/v3/projects/5b0c0c54ba4c4bed9b05212d76ebc4b7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:20.410666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.411211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.411329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.430693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:20.431933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:20.491244 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1338/2676] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:20.500314 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.500594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.500858 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.521241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:20.522420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:20.530027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1339/2677] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:20.534582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.544122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449de11-8591-48f6-9f53-743e998782f0 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.548659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.549243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.549547 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.549825 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.550453 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:20.550784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.552401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.566153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1339/2678] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:20.574199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.584017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c5453-c0a7-4ed8-afeb-3e8c32624124 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.588776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.589578 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.589790 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.590050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.590676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.592598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.683623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1340/2679] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:20 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:20.687726 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.698093 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3a728-42a1-4752-b763-787bb0066b61 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.756427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.757026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.757303 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.757569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.758182 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.760097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.859750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1340/2680] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:20 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:20.864495 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.874884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f312-35c6-4df5-a5cd-c0e9bdfca125 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.880134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.880714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.881050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.881347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.881979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.883618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.893641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1341/2681] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:20.898371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.908301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f38dd-3a6c-4433-bc6e-6f2e96749175 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.913146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.913735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.914012 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.914299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.914941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.916518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c3aa-a914-4f59-8de2-a4cda4170a5c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:20.927827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1341/2682] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:20.931944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:20.943797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9215-cabf-4922-8cf3-c5f5efc5b159 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:20.947206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:20.947840 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:20.948117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:20.948383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] PATH_INFO: `/v3/projects/c5e475549e2e40149631f15c356d763e/users/9f6e9527483d4c659f12f0ce73e7781b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:20.963052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e475549e2e40149631f15c356d763e, user_id=9f6e9527483d4c659f12f0ce73e7781b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:20.965698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.024008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1342/2683] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:20 2022] PUT /identity/v3/projects/c5e475549e2e40149631f15c356d763e/users/9f6e9527483d4c659f12f0ce73e7781b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:21.035974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.035974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.035974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.062503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] MFA Rules not processed for user `9f6e9527483d4c659f12f0ce73e7781b`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:21.063741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:21.119932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:21.130671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1342/2684] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:21.135189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.182641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:21.196657 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ed8e-283f-4600-b464-3f5d1afc93b0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.201342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.201938 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.202216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.202511 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.203167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.205062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.283195 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1343/2685] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:21 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:21.287880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.298415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f86e1-fd89-4983-a1dc-8421dfda543b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.353631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.354230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.354553 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.354858 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.355473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.357606 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.453727 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1343/2686] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:21 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:21.457889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.468618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e5382-62fa-4071-9c0d-c8e3619315f9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.473910 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.474500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.474822 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.475151 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.475838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.480625 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.488545 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1344/2687] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:21.492752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.502841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c8e09-f13b-4489-8d24-e0988a2a0f34 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.507475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.508113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.508389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.508655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] PATH_INFO: `/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a/users/b62e1c63d74848398c154d1c6f6a91a7/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.523420 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab23f6cb363b44bab086e48d7fdf4f3a, user_id=b62e1c63d74848398c154d1c6f6a91a7, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.526056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.613684 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1344/2688] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:21 2022] PUT /identity/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a/users/b62e1c63d74848398c154d1c6f6a91a7/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:21.617617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.627712 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95490-0153-4d67-a6a0-6f0c26f45854 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.684628 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.685215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.685487 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.685754 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.686412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.688047 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc5a13-2125-4caf-8257-30d1a840f49c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.698339 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1345/2689] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:21.702495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.713394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e2f16-5fda-4b36-8459-dbe11b7f6c1d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.717938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.718580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.718893 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.719167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] PATH_INFO: `/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a/users/b62e1c63d74848398c154d1c6f6a91a7/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.721070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab23f6cb363b44bab086e48d7fdf4f3a, user_id=b62e1c63d74848398c154d1c6f6a91a7, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.724320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.734558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1345/2690] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:21 2022] PUT /identity/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a/users/b62e1c63d74848398c154d1c6f6a91a7/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:21.744870 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.745237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.745513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.773261 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] MFA Rules not processed for user `b62e1c63d74848398c154d1c6f6a91a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:21.774448 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:21.841867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1346/2691] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:21.854536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:21.863798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:21.914846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79ba1b-be0d-44b9-9931-845668bbeee4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:21.918492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:21.919157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:21.919435 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:21.919702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:21.929243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:21.930699 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:21.940065 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1346/2692] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:22.716664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:22.726365 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe5475-d4f6-4771-8362-913117749f47 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:22.731378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b62e1c63d74848398c154d1c6f6a91a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:22.732052 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:22.740600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:22.740600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:22.740600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:22.740600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:22.740600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:22.746632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1347/2693] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:22.750905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:22.760706 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96add3-e405-479a-82aa-aca889bef81a service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:22.765532 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b62e1c63d74848398c154d1c6f6a91a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:22.766116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:22.766401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:22.766672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:22.767320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:22.769765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:22.860775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1347/2694] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:22 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:22.865161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:22.875289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df8010-e402-4208-8481-c84b6d2d0f94 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:22.903244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:22.935153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b62e1c63d74848398c154d1c6f6a91a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:22.935806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:22.936085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:22.936356 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:22.936995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:22.937295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:22.938969 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:22.950363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1348/2695] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:22.991915 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:23.002156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263b647-d640-4cb1-a4dc-5171f8b6d058 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:23.006804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b62e1c63d74848398c154d1c6f6a91a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:23.007415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:23.007695 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:23.007963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] PATH_INFO: `/v3/projects/5b952ee5bb7f490cbeeac0167a2e105a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:23.009076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5b952ee5bb7f490cbeeac0167a2e105a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:23.011020 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:23.033831 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:23.105124 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b952ee5bb7f490cbeeac0167a2e105a', 'request_id': 'req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:23.110768 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b952ee5bb7f490cbeeac0167a2e105a', 'request_id': 'req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:23.117917 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b952ee5bb7f490cbeeac0167a2e105a', 'request_id': 'req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:23.179575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] The token cache is being invalidate because project 5b952ee5bb7f490cbeeac0167a2e105a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:23.179862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:23.180501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f3ea2a9f-3180-4f30-a307-17dc34a0ca56 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:23.182035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1348/2696] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:22 2022] DELETE /identity/v3/projects/5b952ee5bb7f490cbeeac0167a2e105a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:23.770950 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:23.772759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:23.838581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d46c2-c83b-40e1-a799-5cb43e0cd119 tempest-VolumeQuotaClassesTest-1294783311 tempest-VolumeQuotaClassesTest-1294783311-project-admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:23.893806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:23.894410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:23.894689 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:23.894993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] PATH_INFO: `/v3/users/9f6e9527483d4c659f12f0ce73e7781b` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:23.896128 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6e9527483d4c659f12f0ce73e7781b)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:23.898038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:24.073456 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6e9527483d4c659f12f0ce73e7781b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.080184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6e9527483d4c659f12f0ce73e7781b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.091376 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:24.092535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6e9527483d4c659f12f0ce73e7781b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.150406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6e9527483d4c659f12f0ce73e7781b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.151026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6e9527483d4c659f12f0ce73e7781b'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.157123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1349/2697] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:23 2022] DELETE /identity/v3/users/9f6e9527483d4c659f12f0ce73e7781b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:24.162277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:24.162787 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:24.162891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:24.181843 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:24.183063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:24.239923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1349/2698] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:24.249992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:24.251396 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:24.307968 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e822d0b-37f4-4870-8f5c-beb8abbb471b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:24.311560 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:24.312143 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:24.312413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:24.312677 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:24.321857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:24.323352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:24.332911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1350/2699] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:24.756879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:24.758900 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:24.770850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5f2c-71d2-4146-b72d-e98f370940f3 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:24.775471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:24.776099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:24.776376 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:24.776666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] PATH_INFO: `/v3/projects/c5e475549e2e40149631f15c356d763e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:24.777789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e475549e2e40149631f15c356d763e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:24.779733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:24.882696 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e475549e2e40149631f15c356d763e', 'request_id': 'req-0a2bd316-0071-4c3f-8112-a894c5eb3c15'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.888194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e475549e2e40149631f15c356d763e', 'request_id': 'req-0a2bd316-0071-4c3f-8112-a894c5eb3c15'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.895054 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e475549e2e40149631f15c356d763e', 'request_id': 'req-0a2bd316-0071-4c3f-8112-a894c5eb3c15'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.990056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] The token cache is being invalidate because project c5e475549e2e40149631f15c356d763e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:24.990366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.991002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:24.992537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1350/2700] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:24 2022] DELETE /identity/v3/projects/c5e475549e2e40149631f15c356d763e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:24.996590 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:25.006393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b427-0d72-46a8-9abf-ddefb99a4a2c service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:25.061041 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:25.061654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:25.061931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:25.062200 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] PATH_INFO: `/v3/users/b62e1c63d74848398c154d1c6f6a91a7` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:25.063359 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b62e1c63d74848398c154d1c6f6a91a7)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:25.065271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:25.239193 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b62e1c63d74848398c154d1c6f6a91a7'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.245943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b62e1c63d74848398c154d1c6f6a91a7'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.257492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b62e1c63d74848398c154d1c6f6a91a7'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.338056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b62e1c63d74848398c154d1c6f6a91a7'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.338674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5ec4a846-abf6-4d3a-ba4f-0ed7e7f010d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b62e1c63d74848398c154d1c6f6a91a7'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.344970 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1351/2701] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:24 2022] DELETE /identity/v3/users/b62e1c63d74848398c154d1c6f6a91a7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:25.769971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:25.773579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:25.785459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd316-0071-4c3f-8112-a894c5eb3c15 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:25.819095 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:25.845196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:25.845808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:25.846258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:25.846570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] PATH_INFO: `/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:25.848077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab23f6cb363b44bab086e48d7fdf4f3a)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:25.849975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:25.985147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'request_id': 'req-d009d4e3-6147-47ee-93e3-f903105b9ccb'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.990868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'request_id': 'req-d009d4e3-6147-47ee-93e3-f903105b9ccb'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:25.997298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab23f6cb363b44bab086e48d7fdf4f3a', 'request_id': 'req-d009d4e3-6147-47ee-93e3-f903105b9ccb'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:26.070755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] The token cache is being invalidate because project ab23f6cb363b44bab086e48d7fdf4f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:26.071050 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:26.071676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-d009d4e3-6147-47ee-93e3-f903105b9ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:26.073169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1351/2702] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:25 2022] DELETE /identity/v3/projects/ab23f6cb363b44bab086e48d7fdf4f3a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:26.077982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.078488 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.078586 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.098182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:26.099422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:26.104908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:26.158861 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1352/2703] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:26.168053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.168343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.168616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.189374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:26.191230 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:26.199309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1352/2704] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:26.203463 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.213279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b666-d238-4ab1-bee9-be43e4bff02c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.217756 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.218337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.218608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.218911 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.219540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:26.219859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.221454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.233383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1353/2705] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:26.238096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.247204 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001f9c-6f25-4ad7-822b-d487ea427dc8 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.251912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.252499 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.252777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.253644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.254280 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.256233 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.334471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1353/2706] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:26.338631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.348543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86301b23-8962-466c-98ef-6239ae7dcab9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.404220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.404811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.405085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.405351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.405965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.407896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.539940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1354/2707] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:26.544208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.556211 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbc636-8b33-4263-8896-2e07eb1a54fe admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.560912 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.561493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.561764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.562026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.562651 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.564246 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.575018 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1354/2708] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:26.578943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.589405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ffc3-a5da-4a00-84fc-0555f2843cca admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.593920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.594505 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.594813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.595084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.595711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.598389 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f778-7bb9-4d58-a760-82afb31e5a27 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.609146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1355/2709] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:26.614524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.624165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91bf00-9c70-4583-a994-828db6096cc2 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.628925 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.629554 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.629824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.630090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] PATH_INFO: `/v3/projects/8ea48fc37d204af992cf0c014bae05c2/users/753e3971cf6b4fd386ff81040fd4d89b/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.644621 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea48fc37d204af992cf0c014bae05c2, user_id=753e3971cf6b4fd386ff81040fd4d89b, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.647286 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.704217 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1355/2710] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:26 2022] PUT /identity/v3/projects/8ea48fc37d204af992cf0c014bae05c2/users/753e3971cf6b4fd386ff81040fd4d89b/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:26.714690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.715816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.715816 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.744744 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] MFA Rules not processed for user `753e3971cf6b4fd386ff81040fd4d89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:26.745924 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:26.809138 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1356/2711] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:26.813757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:26.876492 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1249-347e-46dc-927d-0df3c203a367 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:26.881451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:26.881947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:26.882256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:26.882527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:26.883188 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:26.885092 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:26.889779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:26.996560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1356/2712] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:27.000871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.012522 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9837bc-ff47-4831-b1db-e376bf0cd8a2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.015967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:27.068308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.068925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.069194 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.069454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.070049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.071961 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.167649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1357/2713] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:26 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:27.171615 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.181921 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c440-7a8f-4323-894f-4febf1f07492 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.186654 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.187267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.187545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.187812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.188497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.190070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.200298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1357/2714] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:27.205871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.216455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb223f-1950-460c-a03e-5284223c02db admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.221361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.222462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.222755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.223058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.223690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.225443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c9055-a56b-4508-81a0-f8d64d2a922e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.235411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1358/2715] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:27:27.239493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.249427 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7222-6fc0-4115-909a-30c8ca85247f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.254500 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.255167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.255475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.255744 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] PATH_INFO: `/v3/projects/128abb977b844b5ea81cdf9505afef3a/users/375dffb722854daa88888a02de1367f6/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.271342 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128abb977b844b5ea81cdf9505afef3a, user_id=375dffb722854daa88888a02de1367f6, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.273972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.333786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1358/2716] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:27 2022] PUT /identity/v3/projects/128abb977b844b5ea81cdf9505afef3a/users/375dffb722854daa88888a02de1367f6/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:27.345103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.345390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.345660 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.373852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] MFA Rules not processed for user `375dffb722854daa88888a02de1367f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:27.375074 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:27.438478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1359/2717] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:27.443575 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.505890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71205-bab0-405c-8cac-effc31143fd8 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.510459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.511225 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.511502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.511770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.512380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.514257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.586413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1359/2718] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:27 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:27:27.590564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.601289 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4c72-c165-4a9c-9040-79feeaa42222 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.658233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.658862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.659278 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.659619 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.660271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.662388 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.767318 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1360/2719] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:27 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:27.771374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.781801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002adb-329f-4e8d-9e8e-884a076ba775 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.786668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.787289 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.787566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.787836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.788464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.790144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.801335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1360/2720] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:27.805418 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.811705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:27.816141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334fd1a-603a-4aea-a462-913f0e4de698 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.820886 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.821727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:27.821829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:27.822137 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] PATH_INFO: `/v3/projects/9ae07c4500a44527b5c1e13945fec6eb/users/2757fe96922845baa27bc4ffe974a456/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:27.837758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae07c4500a44527b5c1e13945fec6eb, user_id=2757fe96922845baa27bc4ffe974a456, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:27.840581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:27.927869 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1361/2721] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:27 2022] PUT /identity/v3/projects/9ae07c4500a44527b5c1e13945fec6eb/users/2757fe96922845baa27bc4ffe974a456/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:27.932173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:27.942866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2d964-f435-4ba0-b80e-af35b4d02614 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:27.999215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:27.999817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:28.000121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:28.000394 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:28.001024 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:28.002608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d406bcf-529f-4c39-a16e-455c5a677e60 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:28.012335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1361/2722] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:28.016523 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:28.027204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7feb35-a610-41fb-8add-6527fb18d030 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:28.032644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:28.033240 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:28.033516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:28.033782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] PATH_INFO: `/v3/projects/9ae07c4500a44527b5c1e13945fec6eb/users/2757fe96922845baa27bc4ffe974a456/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:28.036238 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae07c4500a44527b5c1e13945fec6eb, user_id=2757fe96922845baa27bc4ffe974a456, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:28.038884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:28.048918 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1362/2723] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:28 2022] PUT /identity/v3/projects/9ae07c4500a44527b5c1e13945fec6eb/users/2757fe96922845baa27bc4ffe974a456/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:28.059884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:28.060167 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:28.060434 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:28.087447 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] MFA Rules not processed for user `2757fe96922845baa27bc4ffe974a456`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:28.088629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:28.096471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:28.156268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1362/2724] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:28.167340 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:28.168978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:28.227294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04573d5-3add-42f7-85a8-dcdc0a8558fd admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:28.231307 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:28.231896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:28.232168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:28.232432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:28.241357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:28.242810 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:28.252017 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1363/2725] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:29.110666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:29.112099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:29.124002 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523fee-131c-43ea-a656-ad0f7d995bac None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:29.127481 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:29.128366 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:29.128626 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:29.128967 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:29.184029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:29.185456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:29.194488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1363/2726] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:34.063583 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:34.072960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94392ef-b2b7-4751-aaf3-7405898564c2 service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:34.076785 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:34.082434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:34.082434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:34.082434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:34.133317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:34.134768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be17bb2-8f47-43e4-b916-c3477be75e54 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:34.143933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1364/2727] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:38.054001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:38.117456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6d3d6-83d7-4be1-a17d-2a64b899aa91 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:38.122090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:38.122720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:38.123005 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:38.123272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] PATH_INFO: `/v3/users/753e3971cf6b4fd386ff81040fd4d89b` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:38.124372 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753e3971cf6b4fd386ff81040fd4d89b)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:38.126282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:38.310571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:38.312063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753e3971cf6b4fd386ff81040fd4d89b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:38.324249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753e3971cf6b4fd386ff81040fd4d89b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:38.324922 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753e3971cf6b4fd386ff81040fd4d89b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:38.332240 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753e3971cf6b4fd386ff81040fd4d89b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:38.333365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753e3971cf6b4fd386ff81040fd4d89b'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:38.384902 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1364/2728] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:38 2022] DELETE /identity/v3/users/753e3971cf6b4fd386ff81040fd4d89b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:38.389697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:38.390180 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:38.390302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:38.409190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:38.410504 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:38.468587 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1365/2729] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:27:38.478300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:38.479711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:38.538399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cbb9-cfee-45d1-9607-96707b436389 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:38.542297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:38.542978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:38.543256 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:38.543524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:38.553120 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:38.554537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:38.563999 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1365/2730] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:38.969236 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:38.970759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:38.983145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e70c1-1d62-4ee6-b65f-70a0477c2ecd None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:38.988525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:38.989276 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:38.990142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:38.990455 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] PATH_INFO: `/v3/projects/8ea48fc37d204af992cf0c014bae05c2` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:38.991697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea48fc37d204af992cf0c014bae05c2)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:38.993608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:38.996284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:39.126991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea48fc37d204af992cf0c014bae05c2', 'request_id': 'req-816b0ec8-43db-4ce9-801c-15ab22d1678c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.132528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea48fc37d204af992cf0c014bae05c2', 'request_id': 'req-816b0ec8-43db-4ce9-801c-15ab22d1678c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.138554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea48fc37d204af992cf0c014bae05c2', 'request_id': 'req-816b0ec8-43db-4ce9-801c-15ab22d1678c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.200373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] The token cache is being invalidate because project 8ea48fc37d204af992cf0c014bae05c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:39.200659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.201447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.202996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1366/2731] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:38 2022] DELETE /identity/v3/projects/8ea48fc37d204af992cf0c014bae05c2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:39.217305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:39.218165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dd9e0-7a30-4a7f-ad53-d6932e0d3237 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:39.276504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:39.277140 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:39.277415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:39.277683 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] PATH_INFO: `/v3/users/375dffb722854daa88888a02de1367f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:39.279011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375dffb722854daa88888a02de1367f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:39.281037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:39.443330 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375dffb722854daa88888a02de1367f6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.454942 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375dffb722854daa88888a02de1367f6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.455669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375dffb722854daa88888a02de1367f6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.457888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:39.463608 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375dffb722854daa88888a02de1367f6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.464400 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375dffb722854daa88888a02de1367f6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:39.555852 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1366/2732] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:39 2022] DELETE /identity/v3/users/375dffb722854daa88888a02de1367f6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:39.963760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:39.965212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:39.978543 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b0ec8-43db-4ce9-801c-15ab22d1678c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:40.036034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:40.036653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:40.036926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:40.037192 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] PATH_INFO: `/v3/projects/128abb977b844b5ea81cdf9505afef3a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:40.038313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=128abb977b844b5ea81cdf9505afef3a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:40.040257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:40.146029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128abb977b844b5ea81cdf9505afef3a', 'request_id': 'req-5f215d08-d975-4923-af0c-6b1b8b946108'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.151715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128abb977b844b5ea81cdf9505afef3a', 'request_id': 'req-5f215d08-d975-4923-af0c-6b1b8b946108'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.158251 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128abb977b844b5ea81cdf9505afef3a', 'request_id': 'req-5f215d08-d975-4923-af0c-6b1b8b946108'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.245534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] The token cache is being invalidate because project 128abb977b844b5ea81cdf9505afef3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:40.245882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.246509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.248103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1367/2733] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:39 2022] DELETE /identity/v3/projects/128abb977b844b5ea81cdf9505afef3a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:40.252295 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:40.254460 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:40.267214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acf3ce-f204-4f20-865f-8ce28e0c6e00 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:40.323249 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:40.323913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:40.324191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:40.324465 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] PATH_INFO: `/v3/users/2757fe96922845baa27bc4ffe974a456` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:40.325584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2757fe96922845baa27bc4ffe974a456)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:40.327536 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:40.525377 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2757fe96922845baa27bc4ffe974a456'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.537799 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2757fe96922845baa27bc4ffe974a456'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.538496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2757fe96922845baa27bc4ffe974a456'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.545262 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2757fe96922845baa27bc4ffe974a456'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.545881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-dc862c28-2e67-4040-a14e-c43a2ad97641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2757fe96922845baa27bc4ffe974a456'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:40.641169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1367/2734] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:40 2022] DELETE /identity/v3/users/2757fe96922845baa27bc4ffe974a456 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:41.008947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.010393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:41.026186 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f215d08-d975-4923-af0c-6b1b8b946108 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.046597 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:41.084859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.085507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.085812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.086084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] PATH_INFO: `/v3/projects/9ae07c4500a44527b5c1e13945fec6eb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.087214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae07c4500a44527b5c1e13945fec6eb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.089120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.208239 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae07c4500a44527b5c1e13945fec6eb', 'request_id': 'req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:41.213656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae07c4500a44527b5c1e13945fec6eb', 'request_id': 'req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:41.219921 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae07c4500a44527b5c1e13945fec6eb', 'request_id': 'req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:41.287552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] The token cache is being invalidate because project 9ae07c4500a44527b5c1e13945fec6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:41.287862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:41.288475 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-e9bbcb43-4b6c-489c-b44a-8e02c58c79e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:41.289941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1368/2735] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:41 2022] DELETE /identity/v3/projects/9ae07c4500a44527b5c1e13945fec6eb => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:41.297219 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.297687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.297784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.317424 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:41.318624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:41.375981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1368/2736] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:41.385117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.385424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.385699 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.405411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:41.406743 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:41.414797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1369/2737] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:41.419008 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.425042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:41.429001 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10645dcc-da2e-4c01-98c9-e8d630cdd082 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.433821 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.434432 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.434864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.435134 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.435777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:41.436078 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.437665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.449797 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1369/2738] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:41.458308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.465782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6596-3b0d-490c-90f9-57d8fa9c95bb None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.470021 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.470644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.470954 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.471222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.471844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.474113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.569982 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1370/2739] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:41 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:41.574300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.585419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bd5b0-9c32-4e50-90e1-4474c6e11973 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.641594 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.642200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.642480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.642784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.643398 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.645986 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.762129 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1370/2740] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:41 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:41.766510 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.777053 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5db83-8f62-44b4-b823-5f6ef1b2a55d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.781675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.782286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.782562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.782862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.783493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.785063 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.795189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1371/2741] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:41.799007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.809680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1477a-0c75-47cd-aafc-eecb40bc37c9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.814206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.814820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.815097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.815360 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.815980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.818573 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdcc1c-247a-43c1-833b-441097ce6bcc admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.828866 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1371/2742] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:41.832833 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:41.842521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fc070-73a9-4a3d-88b8-2e013900f8dc admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:41.847058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:41.847705 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.847979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.848272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] PATH_INFO: `/v3/projects/ccf02c901d544fb0a4a9bdd9732b4a7c/users/a81e6b9127e9474d93e393431a327204/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:41.862817 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf02c901d544fb0a4a9bdd9732b4a7c, user_id=a81e6b9127e9474d93e393431a327204, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:41.866185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:41.927135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1372/2743] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:41 2022] PUT /identity/v3/projects/ccf02c901d544fb0a4a9bdd9732b4a7c/users/a81e6b9127e9474d93e393431a327204/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:41.994290 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:41.994571 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:41.994923 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.022472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] MFA Rules not processed for user `a81e6b9127e9474d93e393431a327204`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:42.023693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:42.086869 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1372/2744] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:42.091334 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.139780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:42.152931 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38a3cf-bde4-414e-861d-bb3747f279a9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.157623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.158213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.158486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.158779 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.159401 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.161321 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.236844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1373/2745] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:42 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:42.241094 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.251826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106b0f-9b10-4b7e-930f-0756dd19e9e1 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.307449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.308154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.308313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.308578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.309200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.311130 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.319577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:42.430029 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1373/2746] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:42 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:42.434056 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.444979 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c183-d97f-488e-93ef-e1238f89666b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.449531 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.450113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.450381 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.450642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.451352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.452920 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.463737 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1374/2747] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:42.467770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.477837 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce37-17a1-4c97-9649-922555ca0275 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.483147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.483755 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.484035 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.484306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.484943 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.486526 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010a791-a96b-4adf-a125-e6b1a1e5c713 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.496874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1374/2748] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:27:42.500843 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.511485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764b48-c69c-447b-a0aa-be50b43a9a1b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.516148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.516777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.517055 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.517325 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] PATH_INFO: `/v3/projects/399073ccd3a44fd68e9add3c429add85/users/1922ee1aa343472db823b93605e86467/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.531554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=399073ccd3a44fd68e9add3c429add85, user_id=1922ee1aa343472db823b93605e86467, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.534407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.631083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1375/2749] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:42 2022] PUT /identity/v3/projects/399073ccd3a44fd68e9add3c429add85/users/1922ee1aa343472db823b93605e86467/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:42.641436 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.641809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.642144 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.671128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] MFA Rules not processed for user `1922ee1aa343472db823b93605e86467`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:42.672440 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:42.737753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1375/2750] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:42.743363 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.805492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf26ef-8d63-4142-ab15-041c738607ee admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.810520 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.811167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.811451 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.811738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.812370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.814279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:42.917604 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1376/2751] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:42 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:42.921822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:42.933722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43429-8c92-4eb3-9143-1059c51845dc None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:42.990976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:42.991584 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:42.991861 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:42.992148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:42.992757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:42.994675 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.058966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1376/2752] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:42 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:43.062991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.073945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887988b-589f-4eae-ae72-76f6e0fba7cf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.078609 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.079230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.079506 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.079772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.080433 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.082050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.092591 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1377/2753] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:43.096704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.107355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e12058-96f7-4924-85f2-edc39c50572a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.111834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.112469 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.112773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.113006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] PATH_INFO: `/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a/users/50ac72bb193a404db8b8c489ff51b433/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.127876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef81f60a96ce46f89a9eb9ef1e890f1a, user_id=50ac72bb193a404db8b8c489ff51b433, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.130598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.184370 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1377/2754] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:43 2022] PUT /identity/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a/users/50ac72bb193a404db8b8c489ff51b433/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:43.188306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.198471 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c6eb-44fa-40f5-8781-69114a594dfe admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.255184 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.255710 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.255987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.256255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.256877 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.258443 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983db46f-dc6c-4c5a-8243-0abd7a64e383 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.267905 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1378/2755] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:43.272671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.283006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3f1de-8102-43b0-a242-e37ad3e23a83 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.288270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.288904 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.289179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.289451 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] PATH_INFO: `/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a/users/50ac72bb193a404db8b8c489ff51b433/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.291456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef81f60a96ce46f89a9eb9ef1e890f1a, user_id=50ac72bb193a404db8b8c489ff51b433, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.294073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.304287 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1378/2756] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:43 2022] PUT /identity/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a/users/50ac72bb193a404db8b8c489ff51b433/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:43.315698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6008-e651-4736-9b29-24d497254752 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.315989 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6008-e651-4736-9b29-24d497254752 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.316259 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6008-e651-4736-9b29-24d497254752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.335562 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:43.343694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-989c6008-e651-4736-9b29-24d497254752 None None] MFA Rules not processed for user `50ac72bb193a404db8b8c489ff51b433`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:43.344888 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-989c6008-e651-4736-9b29-24d497254752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:43.422876 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1379/2757] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:43.436515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.438450 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:43.464663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:43.500208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc237ebd-9d04-46fc-9a73-a5523d5c3b79 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.503864 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.504563 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.504842 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.505113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.514004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.515410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.525385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1379/2758] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:43.628071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c6008-e651-4736-9b29-24d497254752 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:43.630302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-989c6008-e651-4736-9b29-24d497254752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:43.642254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c6008-e651-4736-9b29-24d497254752 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:43.646828 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:43.647245 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:43.647515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:43.647772 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:43.703736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:43.705162 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8659f-8e79-40b4-84a8-403d35ff7fe5 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:43.715018 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1380/2759] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:47.541971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:47.604480 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136d24-daa9-436a-925d-2b8b735747a8 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:47.609378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:47.609984 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:47.610254 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:47.610519 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] PATH_INFO: `/v3/users/a81e6b9127e9474d93e393431a327204` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:47.611671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81e6b9127e9474d93e393431a327204)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:47.613564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:47.767533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81e6b9127e9474d93e393431a327204'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:47.780304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81e6b9127e9474d93e393431a327204'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:47.780993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81e6b9127e9474d93e393431a327204'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:47.788785 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81e6b9127e9474d93e393431a327204'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:47.789415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81e6b9127e9474d93e393431a327204'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:47.857597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1380/2760] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:47 2022] DELETE /identity/v3/users/a81e6b9127e9474d93e393431a327204 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:47.862067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:47.868286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:47.868286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:47.883750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:47.884929 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:47.943837 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1381/2761] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:47.953300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:47.954670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:47.999485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:48.015807 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c861f1-bcbc-46f1-8aa0-cc01d3557047 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:48.019731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:48.023569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:48.023889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:48.024196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:48.033803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:48.035297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:48.044835 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1381/2762] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:48.402895 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:48.404479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:48.418120 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fb73d-ceb1-4768-a285-d7ebc62a220f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:48.422847 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:48.423461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:48.423731 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:48.423996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] PATH_INFO: `/v3/projects/ccf02c901d544fb0a4a9bdd9732b4a7c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:48.425088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf02c901d544fb0a4a9bdd9732b4a7c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:48.427681 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:48.512574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:48.527826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf02c901d544fb0a4a9bdd9732b4a7c', 'request_id': 'req-a739d906-ef3c-41a5-a4fa-e65d109876d2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.533351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf02c901d544fb0a4a9bdd9732b4a7c', 'request_id': 'req-a739d906-ef3c-41a5-a4fa-e65d109876d2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.540328 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf02c901d544fb0a4a9bdd9732b4a7c', 'request_id': 'req-a739d906-ef3c-41a5-a4fa-e65d109876d2'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.601268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] The token cache is being invalidate because project ccf02c901d544fb0a4a9bdd9732b4a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:48.601616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.602268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.603813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1382/2763] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:48 2022] DELETE /identity/v3/projects/ccf02c901d544fb0a4a9bdd9732b4a7c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:48.608232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:48.618257 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfadd15-ba0f-45a1-9600-12d4a01e4042 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:48.673542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:48.674189 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:48.674472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:48.674767 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] PATH_INFO: `/v3/users/1922ee1aa343472db823b93605e86467` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:48.675889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1922ee1aa343472db823b93605e86467)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:48.678027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:48.820612 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1922ee1aa343472db823b93605e86467'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.832388 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1922ee1aa343472db823b93605e86467'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.833834 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1922ee1aa343472db823b93605e86467'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.840544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1922ee1aa343472db823b93605e86467'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.841166 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1922ee1aa343472db823b93605e86467'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:48.903511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1382/2764] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:48 2022] DELETE /identity/v3/users/1922ee1aa343472db823b93605e86467 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:49.357414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:49.358896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:49.372528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739d906-ef3c-41a5-a4fa-e65d109876d2 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:49.430812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:49.431557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:49.431835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:49.432125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] PATH_INFO: `/v3/projects/399073ccd3a44fd68e9add3c429add85` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:49.433281 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=399073ccd3a44fd68e9add3c429add85)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:49.435214 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:49.538154 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399073ccd3a44fd68e9add3c429add85', 'request_id': 'req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.543675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399073ccd3a44fd68e9add3c429add85', 'request_id': 'req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.549714 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399073ccd3a44fd68e9add3c429add85', 'request_id': 'req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.612211 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] The token cache is being invalidate because project 399073ccd3a44fd68e9add3c429add85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:49.612440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.613033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.615227 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1383/2765] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:49 2022] DELETE /identity/v3/projects/399073ccd3a44fd68e9add3c429add85 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:49.619298 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:49.620750 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:49.633355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5874f-6d5a-49a9-be2d-b7e92345c568 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:49.689680 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:49.690302 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:49.690577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:49.690889 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] PATH_INFO: `/v3/users/50ac72bb193a404db8b8c489ff51b433` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:49.692165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ac72bb193a404db8b8c489ff51b433)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:49.694048 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:49.699033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:49.855115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ac72bb193a404db8b8c489ff51b433'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.867813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ac72bb193a404db8b8c489ff51b433'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.868425 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ac72bb193a404db8b8c489ff51b433'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.875191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ac72bb193a404db8b8c489ff51b433'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.875819 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-661f38ef-4a3b-4b89-8bd6-b0c12cfbb336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ac72bb193a404db8b8c489ff51b433'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:49.948947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1383/2766] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:27:49 2022] DELETE /identity/v3/users/50ac72bb193a404db8b8c489ff51b433 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:27:50.356871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:50.358319 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:50.372661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ef426-5aa0-4c1a-8580-96f1f4ec3d7d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:50.417459 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:50.429887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:50.430548 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.430887 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.431166 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] PATH_INFO: `/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.432320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef81f60a96ce46f89a9eb9ef1e890f1a)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:50.434332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:50.539296 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef81f60a96ce46f89a9eb9ef1e890f1a', 'request_id': 'req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:50.544624 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef81f60a96ce46f89a9eb9ef1e890f1a', 'request_id': 'req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:50.550551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef81f60a96ce46f89a9eb9ef1e890f1a', 'request_id': 'req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:50.646420 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] The token cache is being invalidate because project ef81f60a96ce46f89a9eb9ef1e890f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:27:50.646845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:50.647361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-29a8b699-716b-4be8-b2c1-2e6f4b60c8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:27:50.648835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1384/2767] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:27:50 2022] DELETE /identity/v3/projects/ef81f60a96ce46f89a9eb9ef1e890f1a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:50.654794 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.655282 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.655403 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.675357 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:50.677117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:50.736312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1384/2768] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:50.744606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.744882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.745149 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.765109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:50.766354 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:50.774952 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1385/2769] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:50.779489 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:50.789948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d67fa-7be9-4a4b-8d52-d5123b64fa33 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:50.794611 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:50.795232 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.795510 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.796194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.796413 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:27:50.797336 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:50.799165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:50.811599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1385/2770] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:27:50.815955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:50.826178 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbf4c4-165e-437e-aa4c-328899bfe9b6 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:50.830813 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:50.831398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.831672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.831937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.832595 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:50.834881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:50.916798 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1386/2771] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:50.921378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:50.933011 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54cd64-f78e-4c62-a2ca-c2b374c03919 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:50.987630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:50.988221 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:50.988497 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:50.989296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:50.990040 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:50.992007 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.058624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:51.063713 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1386/2772] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:50 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:51.068410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.078552 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78f3-ef4f-452c-b22d-3e0bf221eb3c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.083350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.083937 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.084215 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.084482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.085144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.086727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.096500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1387/2773] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:51.100911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.111017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a022f-7284-4624-b556-563b6caec0a4 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.116486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.117075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.117356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.117629 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.118265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.119933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b39d-592b-4f80-b1d0-17f5ffb9b613 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.131227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1387/2774] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:51.135567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.145622 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9675e-f943-4005-8c87-cebb59bec585 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.150330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.150991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.151272 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.151541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] PATH_INFO: `/v3/projects/e7c07e63a7c04feb92cb263acd12b1fa/users/ec8ea790d83f4ca283f39cdd5df98f26/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.166561 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c07e63a7c04feb92cb263acd12b1fa, user_id=ec8ea790d83f4ca283f39cdd5df98f26, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.169250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.252685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1388/2775] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:51 2022] PUT /identity/v3/projects/e7c07e63a7c04feb92cb263acd12b1fa/users/ec8ea790d83f4ca283f39cdd5df98f26/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:51.264197 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.264478 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.264748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.293073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] MFA Rules not processed for user `ec8ea790d83f4ca283f39cdd5df98f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:51.294279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:51.357156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1388/2776] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:27:51.361602 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.422458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfc0-e182-4875-9764-4ebb4c75cb0f admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.427935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.428964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.428964 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.429196 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.429653 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.431565 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.510199 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:51.517233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1389/2777] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:27:51 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:51.522069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.532733 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb9b49-1127-404b-8a07-e4d8524d97ba None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.590281 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.590916 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.591194 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.591463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.592077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.593978 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.670066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1389/2778] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:27:51 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:27:51.674100 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.684769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba303d2a-96e1-406b-8a19-9f9e91c143de admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.689526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.690113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.690383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.690644 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.691350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.693257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.703789 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1390/2779] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:51.708689 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.718619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883bfc-0b7d-476b-b1c0-d3b78f24fd98 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.723947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.724661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.724939 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.725210 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] PATH_INFO: `/v3/projects/24f694de3a47415db4484ce40b846caa/users/8bbf3afcabdb432c996f67312c1e2f72/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.740081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f694de3a47415db4484ce40b846caa, user_id=8bbf3afcabdb432c996f67312c1e2f72, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.742798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.811911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1390/2780] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:51 2022] PUT /identity/v3/projects/24f694de3a47415db4484ce40b846caa/users/8bbf3afcabdb432c996f67312c1e2f72/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:51.815800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.826632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9fce-fa28-4d6b-bf14-ccab2a751923 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.882465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.883088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.883362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.883627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.884252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.885827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4ce8f-9793-43e8-997d-c96457783fc8 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.895295 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1391/2781] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:27:51.899810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:51.909913 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1c32-02ae-44d6-94dc-ebd811f69e04 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:51.915657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:51.916292 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.916567 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.916833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] PATH_INFO: `/v3/projects/24f694de3a47415db4484ce40b846caa/users/8bbf3afcabdb432c996f67312c1e2f72/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.918773 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f694de3a47415db4484ce40b846caa, user_id=8bbf3afcabdb432c996f67312c1e2f72, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:51.921393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:51.932176 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1391/2782] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:27:51 2022] PUT /identity/v3/projects/24f694de3a47415db4484ce40b846caa/users/8bbf3afcabdb432c996f67312c1e2f72/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:27:51.943663 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:51.943953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:51.944221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:51.971107 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] MFA Rules not processed for user `8bbf3afcabdb432c996f67312c1e2f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:27:51.972292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:52.039189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1392/2783] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:27:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:27:52.050099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:52.052679 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:52.085871 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:27:52.110548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4a2e-1ae7-457a-a224-76c8bd62e3d5 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:52.115025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:52.115618 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:52.115888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:52.116152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:52.170946 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:52.172364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:52.181756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1392/2784] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:27:57.159419 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:27:57.160908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:27:57.172646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235df619-3d9f-4934-83b6-931466c99d8e None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:27:57.176317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:27:57.176769 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:27:57.177046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:27:57.177317 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:27:57.186096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:27:57.187532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e24a4f-0ca2-464a-a611-3b247c60a854 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:27:57.197596 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1393/2785] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:27:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:28:02.177873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=34 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:02.871600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:02.932625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92982b7d-1564-4ab6-9de4-bd01688df353 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:02.937110 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:02.937728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:02.937996 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:02.938264 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] PATH_INFO: `/v3/users/ec8ea790d83f4ca283f39cdd5df98f26` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:02.939390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8ea790d83f4ca283f39cdd5df98f26)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:02.941315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:03.111777 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8ea790d83f4ca283f39cdd5df98f26'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.123880 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8ea790d83f4ca283f39cdd5df98f26'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.124542 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8ea790d83f4ca283f39cdd5df98f26'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.131809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8ea790d83f4ca283f39cdd5df98f26'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.132431 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8ea790d83f4ca283f39cdd5df98f26'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.183112 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1393/2786] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:28:02 2022] DELETE /identity/v3/users/ec8ea790d83f4ca283f39cdd5df98f26 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:28:03.187615 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:03.188123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:03.188258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:03.207830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:28:03.209926 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:03.257477 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=100 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:03.268576 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1394/2787] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:28:03.278354 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:03.279782 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:03.336665 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76fc7-5780-4e5e-ae6b-7d28059a00d0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:03.340277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:03.340874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:03.341146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:03.341411 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:03.351102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:03.352527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:03.362291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1394/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:28:03.762955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:03.764524 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:03.778933 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b4e1-3bcf-4672-8004-6db40a2a6d4c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:03.783855 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:03.784478 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:03.784753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:03.785023 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] PATH_INFO: `/v3/projects/e7c07e63a7c04feb92cb263acd12b1fa` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:03.786135 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c07e63a7c04feb92cb263acd12b1fa)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:03.792097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:03.881815 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c07e63a7c04feb92cb263acd12b1fa', 'request_id': 'req-750eee8c-835d-4d9f-a864-b4b580d3c771'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.887258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c07e63a7c04feb92cb263acd12b1fa', 'request_id': 'req-750eee8c-835d-4d9f-a864-b4b580d3c771'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.893267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c07e63a7c04feb92cb263acd12b1fa', 'request_id': 'req-750eee8c-835d-4d9f-a864-b4b580d3c771'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.954998 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] The token cache is being invalidate because project e7c07e63a7c04feb92cb263acd12b1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:28:03.955282 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.955990 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:03.957532 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1395/2789] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:28:03 2022] DELETE /identity/v3/projects/e7c07e63a7c04feb92cb263acd12b1fa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:28:03.961541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:03.971438 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2df76-af6c-4899-a44e-94ba3476d698 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:03.990756 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:04.031000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:04.031614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:04.031888 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:04.032173 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] PATH_INFO: `/v3/users/8bbf3afcabdb432c996f67312c1e2f72` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:04.033271 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbf3afcabdb432c996f67312c1e2f72)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:04.035304 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:04.186770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbf3afcabdb432c996f67312c1e2f72'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.199075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbf3afcabdb432c996f67312c1e2f72'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.199745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbf3afcabdb432c996f67312c1e2f72'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.206392 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbf3afcabdb432c996f67312c1e2f72'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.207030 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f55f51c8-b74d-4665-9ac1-9bdf4c86cf01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbf3afcabdb432c996f67312c1e2f72'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.263491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1395/2790] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:28:03 2022] DELETE /identity/v3/users/8bbf3afcabdb432c996f67312c1e2f72 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:28:04.688123 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:04.689618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:04.704992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eee8c-835d-4d9f-a864-b4b580d3c771 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:04.763535 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:04.764150 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:04.764422 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:04.764686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] PATH_INFO: `/v3/projects/24f694de3a47415db4484ce40b846caa` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:04.765800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f694de3a47415db4484ce40b846caa)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:04.767739 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:04.870447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f694de3a47415db4484ce40b846caa', 'request_id': 'req-107cfa17-e711-4afa-810e-7fa7a19e557a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.876248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f694de3a47415db4484ce40b846caa', 'request_id': 'req-107cfa17-e711-4afa-810e-7fa7a19e557a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.882461 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f694de3a47415db4484ce40b846caa', 'request_id': 'req-107cfa17-e711-4afa-810e-7fa7a19e557a'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.954044 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] The token cache is being invalidate because project 24f694de3a47415db4484ce40b846caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:28:04.954253 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.955071 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:28:04.956541 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1396/2791] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:28:04 2022] DELETE /identity/v3/projects/24f694de3a47415db4484ce40b846caa => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:28:14.259770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:14.950941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:28.933203 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:28.935070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:28.935070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:28.953309 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] MFA Rules not processed for user `b5ffbaec36054aa3a89cca8b6007c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:28:28.954687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:29.007764 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1396/2792] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:28:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:28:29.315539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:29.316935 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:29.373944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cfa17-e711-4afa-810e-7fa7a19e557a admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:29.377485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:29.378077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:29.378351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:29.378618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:29.380352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:29.389834 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:29.391286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b599394-2e65-46db-bbe1-9bcf66efb9ab service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:29.399077 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1397/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:28:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:28:39.001229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=25 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:39.395685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=26 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:28:44.569874 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:44.571383 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:44.651226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00a1c-972a-45b7-a33e-ac9722ea813a None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:44.656735 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:44.657529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:44.657663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:44.657881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:44.659786 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:44.673393 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:44.674591 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:44.688439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1397/2794] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:28:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:28:54.680693 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=32 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.121113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.121645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.121645 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.140234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] MFA Rules not processed for user `b5ffbaec36054aa3a89cca8b6007c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:14.141432 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:14.148571 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1398/2795] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:30:21.777691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.787296 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e67f9-a079-450c-b790-dca2e50b428d service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.790908 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.791491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.791766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.792032 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.793720 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:21.863745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.865305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.876137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1398/2796] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:30:23.054784 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.064410 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30907ae-046a-468b-9adc-88b770b9e6e4 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.069323 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.070113 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.070256 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.070516 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.072241 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:23.086658 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.088103 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.098901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1399/2797] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:30:31.871334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=34 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.093995 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:44.950364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:44.959090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6884-499a-402e-88cf-2682bdee73f2 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:44.962635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:44.964075 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:44.964410 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:44.964681 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:44.966363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:44.975833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:44.977227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:44.985371 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1399/2798] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:31:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:31:48.487713 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:48.489129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:48.503069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb659e-0c52-4d96-9204-c36678fc4392 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:48.508753 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:48.509350 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:48.509752 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:48.510097 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:48.511919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:48.524496 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:48.526809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558ec6-538d-4738-8609-6703d3ba19ba service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:48.542793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1400/2799] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:31:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:31:50.324037 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:50.337229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1462719-ed4e-4b95-a269-9d7845736bf0 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:50.343385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:50.343968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:50.344245 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:50.344516 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:50.346293 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:50.356765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:50.358326 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:50.367146 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1400/2800] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:31:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:31:53.665027 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:53.665526 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:53.666217 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:53.686197 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] MFA Rules not processed for user `b5ffbaec36054aa3a89cca8b6007c8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:53.687398 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:53.694051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1401/2801] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:31:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:31:54.400157 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:54.402170 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:54.417265 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a59846-1673-4ada-9d0d-3f5e02f1893f service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:54.422325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:54.422948 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:54.423222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:54.423485 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:54.425148 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:54.438288 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:54.439932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:54.455131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1401/2802] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:31:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:32:03.679872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:04.452386 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=18 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:36.338008 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:36.339435 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:36.352110 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5666f-06ba-4af5-bf34-7cfe2107b3f2 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:36.355642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:36.356246 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:36.356500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:36.356762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:36.358466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:36.421183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:36.422655 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:36.432462 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1402/2803] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:33:39.943355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:39.955320 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a920-f006-4244-ab92-7ac1d05dd6ab service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.960759 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.961705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.961849 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.962043 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.964595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:39.978457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.980096 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.993593 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1402/2804] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:33:40.089953 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:40.091466 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:40.114749 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ed84-a5b7-4cad-9f6c-ead2f738ef37 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:40.133220 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:40.137525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:40.137525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:40.137525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] PATH_INFO: `/v3/users/b5ffbaec36054aa3a89cca8b6007c8bb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:40.137525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ffbaec36054aa3a89cca8b6007c8bb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:40.137525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:40.344474 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ffbaec36054aa3a89cca8b6007c8bb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.351465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ffbaec36054aa3a89cca8b6007c8bb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.363016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ffbaec36054aa3a89cca8b6007c8bb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.408185 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ffbaec36054aa3a89cca8b6007c8bb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.408800 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-351b23dc-1dfa-4ea6-8786-376c70771c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ffbaec36054aa3a89cca8b6007c8bb'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.414943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1403/2805] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:33:40 2022] DELETE /identity/v3/users/b5ffbaec36054aa3a89cca8b6007c8bb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:40.657363 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:40.694845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2d843-6dcd-461a-87f7-f1e622ba8f7d service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:40.753486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:40.754119 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:40.754397 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:40.754669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] PATH_INFO: `/v3/projects/ef103f8a63404f918eaa5dd0c626b4c0` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:40.755855 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef103f8a63404f918eaa5dd0c626b4c0)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:40.757810 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:40.866812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef103f8a63404f918eaa5dd0c626b4c0', 'request_id': 'req-75900992-b06a-4d55-aafd-a62d67c559cd'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.872459 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef103f8a63404f918eaa5dd0c626b4c0', 'request_id': 'req-75900992-b06a-4d55-aafd-a62d67c559cd'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.879258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef103f8a63404f918eaa5dd0c626b4c0', 'request_id': 'req-75900992-b06a-4d55-aafd-a62d67c559cd'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.950974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] The token cache is being invalidate because project ef103f8a63404f918eaa5dd0c626b4c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80544) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:33:40.951628 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.952277 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-75900992-b06a-4d55-aafd-a62d67c559cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:40.953412 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1403/2806] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:33:40 2022] DELETE /identity/v3/projects/ef103f8a63404f918eaa5dd0c626b4c0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:40.958508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:40.958812 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:40.959083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:40.978728 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:40.979960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:41.020826 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:41.039332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1404/2807] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:41.048506 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.048791 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.049059 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.069803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:41.071104 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:41.079508 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1404/2808] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:41.083835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.094024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195369ad-739e-4eaf-bfb9-2bdce33e2635 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.098857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.099485 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.099748 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.100014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.100649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80545) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:41.100943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.102547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.115347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1405/2809] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:41.119770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.129590 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36b32e-36cd-4a64-bd23-3b635908aabd None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.134272 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.134899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.135179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.135449 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.136077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.137975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.223268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1405/2810] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:33:41 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:41.227271 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.237134 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eaf16-7618-429e-abf6-2c76b26e0e6c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.292914 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.293508 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.293791 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.294051 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] PATH_INFO: `/v3/users` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.294750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.296940 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.397233 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1406/2811] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:33:41 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:41.401399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.411975 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85ebe-70ca-4f41-9f76-f5dba7304eaf admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.416644 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.417234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.417515 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.417784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.418422 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.420063 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.429828 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1406/2812] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:33:41.433902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.444648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d3308-9f06-4282-af04-2d0e6531fbcf admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.449411 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.449993 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.450286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.450537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.451201 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.453649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916dd82-b89e-4dd1-8d49-982e1c7278d5 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.466563 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1407/2813] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:33:41.471165 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.481343 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842509e-eb47-4f52-a43b-00d085af92d7 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.485972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.486610 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.486919 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.487226 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] PATH_INFO: `/v3/projects/6c2fbf0067784cec85fe7ac85967dc27/users/707b86df9efa46c99f17c594ee035c6e/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.499994 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:41.503993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2fbf0067784cec85fe7ac85967dc27, user_id=707b86df9efa46c99f17c594ee035c6e, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.506813 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.586527 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1407/2814] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:33:41 2022] PUT /identity/v3/projects/6c2fbf0067784cec85fe7ac85967dc27/users/707b86df9efa46c99f17c594ee035c6e/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:41.597517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.597835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.598116 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.626944 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] MFA Rules not processed for user `707b86df9efa46c99f17c594ee035c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:41.628058 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:41.691191 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1408/2815] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:41.702454 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.706175 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:41.778841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d487d-64bf-41d4-846f-1b9642d11bce admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:41.783268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:41.783857 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.784131 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.784395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.792479 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:41.793895 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:41.803034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1408/2816] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:33:45.599465 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:45.600862 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:45.647513 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:45.696680 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6508-a649-4be0-8337-acb0253e3aa0 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:45.702515 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:45.703142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:45.703421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:45.703690 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:45.706226 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:45.719849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:45.721692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb08e6-2dac-409b-a184-8eab9c8ac459 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:45.731947 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1409/2817] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:33:51.579331 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.641088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019f323-3183-47c1-8385-46ab873da7e6 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.645595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.646215 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.646494 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.646801 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] PATH_INFO: `/v3/users/707b86df9efa46c99f17c594ee035c6e` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.647914 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=707b86df9efa46c99f17c594ee035c6e)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.649833 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.805971 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707b86df9efa46c99f17c594ee035c6e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:51.818585 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707b86df9efa46c99f17c594ee035c6e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:51.819351 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707b86df9efa46c99f17c594ee035c6e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:51.826443 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707b86df9efa46c99f17c594ee035c6e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:51.827098 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707b86df9efa46c99f17c594ee035c6e'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:51.830657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.901441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1409/2818] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:33:51 2022] DELETE /identity/v3/users/707b86df9efa46c99f17c594ee035c6e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:51.906651 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.907190 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.907400 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.926648 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:51.927856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:51.986962 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1410/2819] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:51.997066 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:52.001114 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:52.063074 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5e33e-79dd-4f9a-911c-dbf5d87dc7f0 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.067659 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.068545 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.068771 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.069031 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.079905 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.081334 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52331d20-55e9-4656-ac61-ffd68956a5aa service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.091090 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1410/2820] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:33:52.480036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:52.486605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:52.494827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257089bd-f969-41ff-a390-45dffdbe060d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.500360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.500992 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.501736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.501736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] PATH_INFO: `/v3/projects/6c2fbf0067784cec85fe7ac85967dc27` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.502639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2fbf0067784cec85fe7ac85967dc27)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.504623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.608182 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2fbf0067784cec85fe7ac85967dc27', 'request_id': 'req-2fdecad3-5f6c-49be-a62a-2042702f68e3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:52.613793 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2fbf0067784cec85fe7ac85967dc27', 'request_id': 'req-2fdecad3-5f6c-49be-a62a-2042702f68e3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:52.619996 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2fbf0067784cec85fe7ac85967dc27', 'request_id': 'req-2fdecad3-5f6c-49be-a62a-2042702f68e3'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:52.710142 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] The token cache is being invalidate because project 6c2fbf0067784cec85fe7ac85967dc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:33:52.710424 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:52.711104 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2fdecad3-5f6c-49be-a62a-2042702f68e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:33:52.712675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1411/2821] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:33:52 2022] DELETE /identity/v3/projects/6c2fbf0067784cec85fe7ac85967dc27 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:52.717582 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.718101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.718101 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.747955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:52.749766 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:52.808599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1411/2822] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:52.818534 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.818871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.819146 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.840553 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:52.841809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:52.850308 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1412/2823] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:52.854348 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:52.864933 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcec75-9c79-4cce-b8c8-8aa1916e473b None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.869598 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.870191 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.870472 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.870775 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.871426 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:52.871753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.873941 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.885589 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1412/2824] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:52.890229 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:52.899337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb5f2a-334c-4a32-be33-f1201e37bc75 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.903894 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.904618 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.905112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.905414 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.906022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.907958 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.992975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1413/2825] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:33:52 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:52.997208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.007938 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ec9c5-6b60-4763-a3d8-32300aa9044d admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.063887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.064518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.064793 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.065060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.065668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.067600 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.152242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1413/2826] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:33:52 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:53.157005 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.163598 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:53.167254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8268d-453e-4b29-ae6c-1ecddad8988c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.172126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.172715 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.172994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.173267 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.173902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.175547 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.186172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1414/2827] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:33:53.190115 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.200270 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d40b-3fcf-4f3c-b890-5eabdd38eb2c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.205729 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.207373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.207373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.207373 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.207704 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.209081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137ddc2-737f-4889-8a63-53d32545f271 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.210362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:53.221523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1414/2828] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:33:53.225836 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.236502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315543a3-8e4b-424b-881b-76a50130a158 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.241301 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.241943 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.242234 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.242528 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] PATH_INFO: `/v3/projects/effe6fffd1e24d49bd56a51f876e7a6c/users/d99fcb6876a6415191774f8b21eb127c/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.257408 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=effe6fffd1e24d49bd56a51f876e7a6c, user_id=d99fcb6876a6415191774f8b21eb127c, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.260094 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.369085 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1415/2829] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:33:53 2022] PUT /identity/v3/projects/effe6fffd1e24d49bd56a51f876e7a6c/users/d99fcb6876a6415191774f8b21eb127c/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:53.380214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.380495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.380761 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.408317 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] MFA Rules not processed for user `d99fcb6876a6415191774f8b21eb127c`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:53.409441 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:53.472758 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1415/2830] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:53.477250 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.537109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760270-f9c3-464b-a1c0-67f3504b63d1 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.541738 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.542322 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.542593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.542893 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.543509 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.545980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.640049 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1416/2831] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:33:53 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:53.644199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.655475 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff3857-491b-4f0d-9a5d-32d2fab9239d None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.711025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.711548 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.711826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.712097 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.712711 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.714630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.833006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1416/2832] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:33:53 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:53.837497 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.849159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e5ce-2da0-4c31-9ad2-610c95a6150b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.853573 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.854155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.854427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.854694 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.855352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.856925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.868254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1417/2833] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:33:53.872375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:53.882687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e4397-96d9-45b4-a05b-761114fce9a3 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:53.887592 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:53.888283 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:53.889235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:53.889599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] PATH_INFO: `/v3/projects/c767777897f7481aa0b9f3d5b601ffb3/users/b84f8b5cf0c74b0ca53dbafff472940f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:53.905156 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c767777897f7481aa0b9f3d5b601ffb3, user_id=b84f8b5cf0c74b0ca53dbafff472940f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:53.907823 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:53.999305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1417/2834] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:33:53 2022] PUT /identity/v3/projects/c767777897f7481aa0b9f3d5b601ffb3/users/b84f8b5cf0c74b0ca53dbafff472940f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:54.003288 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.013332 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982215-4382-4c3e-98ff-d6557ef2ed0c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.067999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.068593 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.068872 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.069144 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.069805 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.071427 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1e24c-3ef4-40fb-8bc8-04a9906dd19a admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.081627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1418/2835] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:33:54.085798 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.096661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691f060-f081-42ff-b78a-0b6533937af9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.101311 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.101951 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.102222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.102486 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] PATH_INFO: `/v3/projects/c767777897f7481aa0b9f3d5b601ffb3/users/b84f8b5cf0c74b0ca53dbafff472940f/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.104385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c767777897f7481aa0b9f3d5b601ffb3, user_id=b84f8b5cf0c74b0ca53dbafff472940f, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.107033 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.117023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1418/2836] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:33:54 2022] PUT /identity/v3/projects/c767777897f7481aa0b9f3d5b601ffb3/users/b84f8b5cf0c74b0ca53dbafff472940f/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:33:54.129046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.129355 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.129627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.157750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] MFA Rules not processed for user `b84f8b5cf0c74b0ca53dbafff472940f`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:54.158951 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:54.188255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:54.226717 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1419/2837] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:33:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:33:54.237892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.239658 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:54.282234 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:54.297927 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a9ac-9786-4108-bd93-80bae446ae8a admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.301645 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.302237 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.302512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.302817 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.312892 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.314318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.323648 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1419/2838] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:33:54.548486 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.558444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00deb7b-d8d5-43cf-ac30-2c3a3fb1a80b None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.562782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.563371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.563646 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.564357 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.625378 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.626830 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6563cea-4a98-4796-98c4-6ed4d63dfa49 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.635623 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1420/2839] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:04.335016 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=9 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:04.631546 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=44 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:07.303421 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:07.384051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475c37-027c-4875-94f9-8166ff0b9c88 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:07.388676 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:07.389305 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:07.389595 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:07.389945 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] PATH_INFO: `/v3/users/d99fcb6876a6415191774f8b21eb127c` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:07.392107 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d99fcb6876a6415191774f8b21eb127c)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:07.394041 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:07.557705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd99fcb6876a6415191774f8b21eb127c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:07.570375 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd99fcb6876a6415191774f8b21eb127c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:07.571139 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd99fcb6876a6415191774f8b21eb127c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:07.578655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd99fcb6876a6415191774f8b21eb127c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:07.579318 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd99fcb6876a6415191774f8b21eb127c'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:07.660561 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1420/2840] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:34:07 2022] DELETE /identity/v3/users/d99fcb6876a6415191774f8b21eb127c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:34:07.665126 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:07.665616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:07.665936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:07.699698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:07.700892 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:07.759963 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1421/2841] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:07.769847 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:07.771322 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:07.836970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23afe6-3180-46cf-b010-14a696ecc75e admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:07.840614 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:07.841218 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:07.841496 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:07.841770 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:07.849620 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:07.851881 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:07.853335 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:07.863666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1421/2842] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:08.267967 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:08.269405 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:08.282850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3e34d-4e6f-4587-b1f3-f90fa99d45f3 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:08.287711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:08.288368 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:08.288673 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:08.288946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] PATH_INFO: `/v3/projects/effe6fffd1e24d49bd56a51f876e7a6c` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:08.290095 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=effe6fffd1e24d49bd56a51f876e7a6c)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:08.292174 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:08.380172 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'effe6fffd1e24d49bd56a51f876e7a6c', 'request_id': 'req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.385922 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'effe6fffd1e24d49bd56a51f876e7a6c', 'request_id': 'req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.392148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'effe6fffd1e24d49bd56a51f876e7a6c', 'request_id': 'req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.459930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] The token cache is being invalidate because project effe6fffd1e24d49bd56a51f876e7a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:34:08.460212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.460844 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.462346 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1422/2843] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:34:08 2022] DELETE /identity/v3/projects/effe6fffd1e24d49bd56a51f876e7a6c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:34:08.467691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:08.477587 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3fa2-ce3b-41d4-ae60-02bc49483781 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:08.534926 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:08.535537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:08.535809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:08.536073 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] PATH_INFO: `/v3/users/b84f8b5cf0c74b0ca53dbafff472940f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:08.537159 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84f8b5cf0c74b0ca53dbafff472940f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:08.539099 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:08.697000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84f8b5cf0c74b0ca53dbafff472940f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.709518 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84f8b5cf0c74b0ca53dbafff472940f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.710185 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84f8b5cf0c74b0ca53dbafff472940f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.717042 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84f8b5cf0c74b0ca53dbafff472940f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.717668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-5bbfd46a-b55e-43fa-aabf-4836a93a3de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84f8b5cf0c74b0ca53dbafff472940f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:08.820147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1422/2844] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:34:08 2022] DELETE /identity/v3/users/b84f8b5cf0c74b0ca53dbafff472940f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:34:09.231799 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:09.233352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:09.247852 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7fb9-086f-4ea2-b7e4-ac7cb6ae75ec admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:09.306016 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:09.306632 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.306942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.307212 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] PATH_INFO: `/v3/projects/c767777897f7481aa0b9f3d5b601ffb3` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.308351 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c767777897f7481aa0b9f3d5b601ffb3)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:09.310252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:09.326050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:09.444255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c767777897f7481aa0b9f3d5b601ffb3', 'request_id': 'req-126a0dab-065f-48a4-b8bd-182399cb4606'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:09.449507 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c767777897f7481aa0b9f3d5b601ffb3', 'request_id': 'req-126a0dab-065f-48a4-b8bd-182399cb4606'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:09.455809 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c767777897f7481aa0b9f3d5b601ffb3', 'request_id': 'req-126a0dab-065f-48a4-b8bd-182399cb4606'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:09.532777 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] The token cache is being invalidate because project c767777897f7481aa0b9f3d5b601ffb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:34:09.532980 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:09.533592 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-126a0dab-065f-48a4-b8bd-182399cb4606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:34:09.535155 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1423/2845] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:34:09 2022] DELETE /identity/v3/projects/c767777897f7481aa0b9f3d5b601ffb3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:34:09.540025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.540503 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.540625 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.561077 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:09.562247 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:09.624538 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1423/2846] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:09.639230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.639512 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.639776 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.659666 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:09.660845 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:09.668659 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1424/2847] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:09.673423 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:09.683017 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99ccdd-6ba6-4695-87c6-f339bd7c5171 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:09.687623 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:09.688208 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.688602 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.688873 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.689596 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:34:09.689894 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:09.691533 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:09.702686 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1424/2848] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:34:09.707039 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:09.716067 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030441ff-7672-4b36-8734-9a70247b0b66 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:09.721638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:09.722244 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.722518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.722820 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.723440 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:09.725320 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:09.825839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1425/2849] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:34:09 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:34:09.829972 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:09.835243 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:09.840668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad618-3656-40a7-bcf4-0a252beee012 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:09.896890 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:09.897484 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:09.897760 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:09.898027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:09.898635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:09.900631 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:09.987512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1425/2850] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:34:09 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:34:09.992221 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:10.003675 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84420773-eaf2-4285-a860-aa42e12c56b6 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:10.008362 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:10.008949 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:10.009224 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:10.009490 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:10.010121 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:10.011811 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:10.021698 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1426/2851] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:34:10.025718 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:10.035886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1106-8b73-4011-bb53-469a92f86eef admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:10.040327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:10.040924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:10.041198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:10.041463 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:10.042178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:10.043688 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15cf44-d919-4e10-94eb-16c5697a406e admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:10.054728 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1426/2852] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:34:10.058642 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:10.069406 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443b5d-e853-4cb0-86be-20cb3fe46123 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:10.074163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:10.074829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:10.075111 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:10.075383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] PATH_INFO: `/v3/projects/4f77a082a0324845b34bf0bbb33bc8bb/users/32099f8214de4f5caebd0a68efdfb87f/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:10.089848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f77a082a0324845b34bf0bbb33bc8bb, user_id=32099f8214de4f5caebd0a68efdfb87f, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:10.092500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:10.157656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1427/2853] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:34:10 2022] PUT /identity/v3/projects/4f77a082a0324845b34bf0bbb33bc8bb/users/32099f8214de4f5caebd0a68efdfb87f/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:34:10.162668 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ee2bd-52c3-455d-b21b-0f3e80f432c5 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:10.168352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ee2bd-52c3-455d-b21b-0f3e80f432c5 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:10.168352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ee2bd-52c3-455d-b21b-0f3e80f432c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:10.182178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-141ee2bd-52c3-455d-b21b-0f3e80f432c5 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:10.183389 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-141ee2bd-52c3-455d-b21b-0f3e80f432c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:10.241638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1427/2854] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:10.251383 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:10.253072 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:10.309692 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a4c9-d091-47a5-9736-4c3ec2a7bf9b admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:10.313390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:10.314011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:10.314286 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:10.314556 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:10.323568 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:10.324994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:10.334341 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1428/2855] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:11.599558 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2cc54-e8af-42c3-8de1-4a2f742c0794 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:11.600087 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2cc54-e8af-42c3-8de1-4a2f742c0794 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:11.600201 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2cc54-e8af-42c3-8de1-4a2f742c0794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:11.632072 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-7ce2cc54-e8af-42c3-8de1-4a2f742c0794 None None] MFA Rules not processed for user `32099f8214de4f5caebd0a68efdfb87f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:11.633279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7ce2cc54-e8af-42c3-8de1-4a2f742c0794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:11.699812 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1428/2856] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:11.711664 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:11.713273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:11.761306 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:11.773421 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17f5d-2ca1-4a1d-8c9d-f94c8a69c53f service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:11.777237 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:11.777848 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:11.778129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:11.778395 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:11.788176 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:11.789656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:11.800457 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1429/2857] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:13.494911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:13.495439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:13.495439 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:13.514607 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] MFA Rules not processed for user `32099f8214de4f5caebd0a68efdfb87f`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:13.515804 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:13.523049 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1429/2858] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:13.755360 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:13.764669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae259e-9042-4f13-92f0-f3af091df33f service cinder] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:13.768407 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:13.768984 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:13.769274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:13.769521 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:13.771273 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:13.780608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:13.782024 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c12938-ed98-41ed-9129-7420f082b1d8 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:13.790633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1430/2859] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:23.375792 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:23.377657 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:23.382213 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:23.492227 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ce28-183e-43df-9d8b-2cd634d1536e None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:23.498236 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:23.498879 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:23.499572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:23.499572 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:23.501868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:23.515789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:23.517380 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:23.529076 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1430/2860] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:23.787062 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=13 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:30.426630 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c0d64-f9d6-463b-b969-2617341a7d58 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:30.427255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c0d64-f9d6-463b-b969-2617341a7d58 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:30.427255 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c0d64-f9d6-463b-b969-2617341a7d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:30.446387 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-978c0d64-f9d6-463b-b969-2617341a7d58 None None] MFA Rules not processed for user `32099f8214de4f5caebd0a68efdfb87f`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:30.447608 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-978c0d64-f9d6-463b-b969-2617341a7d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:30.454390 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1431/2861] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:34:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:30.689638 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:30.699036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3232a-55fb-424a-8dff-056df1eefc0e service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:30.702564 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:30.703172 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:30.703442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:30.703707 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:30.705957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:30.728672 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:30.730117 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:30.738216 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1431/2862] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:34:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:34:40.441145 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:40.734399 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=55 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:53.678483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec0ba-ecb8-4ecf-80ec-fe4b9b4cc5fc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:53.679050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec0ba-ecb8-4ecf-80ec-fe4b9b4cc5fc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:53.679050 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec0ba-ecb8-4ecf-80ec-fe4b9b4cc5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:53.708909 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a8aec0ba-ecb8-4ecf-80ec-fe4b9b4cc5fc None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:53.710086 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a8aec0ba-ecb8-4ecf-80ec-fe4b9b4cc5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:53.764342 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1432/2863] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Aug 9 18:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:34:53.916137 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:53.917551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:53.988060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f616-424b-49d0-91ac-52afb92aa102 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:53.993047 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:53.993639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:53.993911 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:53.994179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:54.001989 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:54.003617 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:54.012027 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1432/2864] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:35:03.757797 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=29 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:04.009085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=32 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:09.450965 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:09.451438 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:09.451551 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:09.471472 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] MFA Rules not processed for user `7c28473ae0554890a2b55941c15e94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:09.472696 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:09.479823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1433/2865] 10.222.0.36 () {58 vars in 967 bytes} [Tue Aug 9 18:35:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:09.492566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:09.503121 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5306a1-19bd-4bd7-8e15-fdc00a038079 service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:09.508963 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c61ea9321f94455afe35b931b9fcf8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:09.509544 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:09.509846 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:09.510116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:09.517899 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:09.519498 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:09.527344 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1433/2866] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Aug 9 18:35:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:35:12.229539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:12.231299 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:12.300483 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f59dd-0b0c-480e-842a-65d622162dbc None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:12.305567 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:12.306163 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:12.306437 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:12.306727 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:12.308530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:12.320517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:12.322093 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:12.332189 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1434/2867] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:12.364765 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:12.373820 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5709d2c-1b86-4994-97c8-f19862cc5b50 service swift] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:12.378660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:12.379529 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:12.379809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:12.380701 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] PATH_INFO: `/v3/users/32099f8214de4f5caebd0a68efdfb87f` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:12.381882 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32099f8214de4f5caebd0a68efdfb87f)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:12.383976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:12.558084 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32099f8214de4f5caebd0a68efdfb87f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:12.570790 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32099f8214de4f5caebd0a68efdfb87f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:12.571473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32099f8214de4f5caebd0a68efdfb87f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:12.578279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32099f8214de4f5caebd0a68efdfb87f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:12.578955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-fbbd02c6-2a7d-4a86-bc55-8f955749d293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32099f8214de4f5caebd0a68efdfb87f'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:12.641183 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1434/2868] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:35:12 2022] DELETE /identity/v3/users/32099f8214de4f5caebd0a68efdfb87f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:12.900973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:12.912446 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2c3b7-1eac-4439-bede-8796fc3b95f7 service placement] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:12.971248 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:12.971871 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:12.972148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:12.972416 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] PATH_INFO: `/v3/projects/4f77a082a0324845b34bf0bbb33bc8bb` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:12.973517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f77a082a0324845b34bf0bbb33bc8bb)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:12.975498 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.122084 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f77a082a0324845b34bf0bbb33bc8bb', 'request_id': 'req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:13.127518 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f77a082a0324845b34bf0bbb33bc8bb', 'request_id': 'req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:13.134638 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f77a082a0324845b34bf0bbb33bc8bb', 'request_id': 'req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:13.206736 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] The token cache is being invalidate because project 4f77a082a0324845b34bf0bbb33bc8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:35:13.206941 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:13.207570 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-63e29de5-bcf6-4eca-8f84-c03773a3ba2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:13.209133 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1435/2869] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:35:12 2022] DELETE /identity/v3/projects/4f77a082a0324845b34bf0bbb33bc8bb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:13.214541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.214957 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.215238 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.262577 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:13.263808 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:13.322966 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1435/2870] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:13.331540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.332060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.332060 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.353037 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:13.354222 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:13.361930 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1436/2871] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:13.367200 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.376741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a85079-51da-48e3-9b3f-ed0311740a10 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.382085 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.382677 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.383070 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.383374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.383974 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:35:13.384297 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.385877 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.397325 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1436/2872] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:35:13.401639 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.410806 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18622-d1d2-49e6-ba9e-2496ee5d8089 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.415755 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.416337 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.416613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.416881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.417492 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.419444 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.423882 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:13.515762 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1437/2873] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:35:13 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:13.519940 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.529980 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c629f3-c988-41ed-8b72-bc905f936447 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.550069 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:13.586907 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.587511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.587788 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.588056 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.588669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.590570 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.691187 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1437/2874] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:35:13 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:13.695370 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.705898 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710626-7b46-4be2-a04f-848ee87391bb admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.710794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.711382 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.711656 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.711919 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.712566 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.714136 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.724780 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1438/2875] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:13.730034 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.740712 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feb06d-68e4-48f2-9ce2-c29a4d4a2324 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.745350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.745947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.746207 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.746474 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.747151 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.748710 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e153c47-0e66-43f9-a594-dcc6481890f9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.758501 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1438/2876] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:13.762452 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.772158 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35768a-cf9b-474b-9fae-a230c64043d9 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:13.776740 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:13.777361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.777634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.777900 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] PATH_INFO: `/v3/projects/2d6530724bd14cffb27a4b667898c85e/users/702c6937861b4e78b03284631d9bcbc6/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.792454 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6530724bd14cffb27a4b667898c85e, user_id=702c6937861b4e78b03284631d9bcbc6, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:13.795108 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:13.870758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1439/2877] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:35:13 2022] PUT /identity/v3/projects/2d6530724bd14cffb27a4b667898c85e/users/702c6937861b4e78b03284631d9bcbc6/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:13.880671 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:13.880949 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:13.883473 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:13.909419 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] MFA Rules not processed for user `702c6937861b4e78b03284631d9bcbc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:13.912108 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:13.975964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1439/2878] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:13.987216 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:13.988889 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:14.066125 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5c2dd-df24-4ddf-aa32-ddd936d209e0 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:14.070685 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:14.071300 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:14.071572 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:14.071838 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:14.079932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:14.081371 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065c83c-3097-4daf-9807-755f7e0ed319 service cinder] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:14.090277 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1440/2879] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:14.115747 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:14.176329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e41675-6c7a-4b6e-9394-36510d6a17eb None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:14.181517 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:14.182127 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:14.182401 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:14.182666 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] PATH_INFO: `/v3/users/702c6937861b4e78b03284631d9bcbc6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:14.183809 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702c6937861b4e78b03284631d9bcbc6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:14.185703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:14.375579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:14.384836 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702c6937861b4e78b03284631d9bcbc6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:14.396970 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702c6937861b4e78b03284631d9bcbc6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:14.397653 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702c6937861b4e78b03284631d9bcbc6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:14.404307 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702c6937861b4e78b03284631d9bcbc6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:14.407661 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702c6937861b4e78b03284631d9bcbc6'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:14.481678 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1440/2880] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:35:14 2022] DELETE /identity/v3/users/702c6937861b4e78b03284631d9bcbc6 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:14.486204 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:14.486729 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:14.486853 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:14.506347 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:14.507637 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:14.565161 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1441/2881] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:14.576493 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:14.577990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:14.634826 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3ce90-1e86-45f1-b6f9-304ebf5aac39 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:14.638378 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:14.639006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:14.639285 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:14.639556 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:14.649062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:14.650488 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5186c-6b70-4eeb-b28e-34f2362f91c2 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:14.659753 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1441/2882] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:15.288902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:15.290327 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:15.302808 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6069e1-c0e4-4d7d-9a05-af23fdc93f42 None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:15.307634 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:15.308258 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.308530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.308823 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] PATH_INFO: `/v3/projects/2d6530724bd14cffb27a4b667898c85e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.309946 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6530724bd14cffb27a4b667898c85e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:15.311904 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:15.409775 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:15.424672 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6530724bd14cffb27a4b667898c85e', 'request_id': 'req-060770b5-0034-438f-bb12-9640091761fc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:15.430928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6530724bd14cffb27a4b667898c85e', 'request_id': 'req-060770b5-0034-438f-bb12-9640091761fc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:15.437069 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6530724bd14cffb27a4b667898c85e', 'request_id': 'req-060770b5-0034-438f-bb12-9640091761fc'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:15.501616 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] The token cache is being invalidate because project 2d6530724bd14cffb27a4b667898c85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:35:15.502269 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:15.502594 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-060770b5-0034-438f-bb12-9640091761fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:35:15.504083 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1442/2883] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:35:15 2022] DELETE /identity/v3/projects/2d6530724bd14cffb27a4b667898c85e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:15.509990 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.510470 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.510586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.529876 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:15.531102 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:15.589977 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1442/2884] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:15.597945 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.598230 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.598502 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.618925 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:15.620112 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:15.627978 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1443/2885] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:15.632910 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:15.642306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ccbf6-467b-4845-b069-a700f69280c6 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:15.646891 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:15.647504 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.647789 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.648051 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] PATH_INFO: `/v3/domains` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.648674 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80544) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:35:15.648964 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:15.650525 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:15.661685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1443/2886] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Aug 9 18:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:35:15.665948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:15.675148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cbb54-d6d5-445f-af0f-89ee018ce97d None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:15.680403 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:15.680986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.681262 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.681529 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.682139 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:15.684080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:15.779801 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1444/2887] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:35:15 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:15.789456 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:15.799682 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75e6a1-d0cc-4ecd-9377-3ba14bd0393b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:15.839685 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:15.856541 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:15.857141 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:15.857428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:15.857697 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:15.858312 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:15.860260 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:15.990779 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1444/2888] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:35:15 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:15.994818 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:16.004912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02334f6-ce50-42ef-a8da-4b214f5ec013 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:16.009540 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:16.010122 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:16.010415 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:16.010686 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:16.011393 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:16.013032 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:16.022758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1445/2889] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:16.026566 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:16.037731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b988086-7675-4b1f-a7db-db58c473bf23 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:16.042448 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:16.043053 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:16.043329 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:16.043597 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:16.044223 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:16.045955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b708d-98f4-45e6-a622-a41653ef7040 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:16.063802 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1445/2890] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:16.068183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:16.078175 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f35c-5f1d-40d9-b815-2c8cfe643811 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:16.082654 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:16.083372 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:16.083649 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:16.083927 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] PATH_INFO: `/v3/projects/13bae50787744ee98e555d0db03e7c0e/users/ac1260c6ff5943f4b4e86ea91f232452/roles/8e6f9bbf4e854915a68a5b685ccb788d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:16.099364 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bae50787744ee98e555d0db03e7c0e, user_id=ac1260c6ff5943f4b4e86ea91f232452, role_id=8e6f9bbf4e854915a68a5b685ccb788d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:16.102014 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:16.168092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1446/2891] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:35:16 2022] PUT /identity/v3/projects/13bae50787744ee98e555d0db03e7c0e/users/ac1260c6ff5943f4b4e86ea91f232452/roles/8e6f9bbf4e854915a68a5b685ccb788d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:16.173301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756c1c3-c69b-44f7-8f7a-91b53f062ede None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:16.173579 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756c1c3-c69b-44f7-8f7a-91b53f062ede None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:16.173851 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756c1c3-c69b-44f7-8f7a-91b53f062ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:16.193537 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-4756c1c3-c69b-44f7-8f7a-91b53f062ede None None] MFA Rules not processed for user `c800e45fdce14a59ae5bf0f84f3671cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:16.194778 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-4756c1c3-c69b-44f7-8f7a-91b53f062ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:16.253408 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1446/2892] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:35:16.263352 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:16.265096 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:16.321252 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e6bf7-81d8-4bb4-8672-dc9fad8b0b70 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:16.324859 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:16.325447 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:16.325720 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:16.325987 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:16.335336 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:16.336758 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:16.346203 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1447/2893] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:21.215345 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.216235 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.216511 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.247635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.auth.core [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] MFA Rules not processed for user `ac1260c6ff5943f4b4e86ea91f232452`. Rule list: `[]` (Enabled: `True`). {{(pid=80544) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:21.248924 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:21.319045 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1447/2894] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:21.323724 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.333986 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b745-12d4-4bec-bb12-5a8f97b67944 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.339038 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.339674 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.339948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.340213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.340839 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.343033 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.437088 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1448/2895] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Aug 9 18:35:21 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:21.441352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.458126 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb864bfc-db28-4acc-981d-d8a584ccf2a7 None None] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.509457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:21.519116 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.519745 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] REQUEST_METHOD: `POST` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.520026 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.520299 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] PATH_INFO: `/v3/users` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.520930 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.522887 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.589722 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1448/2896] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Aug 9 18:35:21 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:35:21.594283 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.605661 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ab00-7af4-44d4-968a-221da7b37e7e admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.610254 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.610875 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.611148 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.611413 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.612036 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.613605 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.624901 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1449/2897] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:21.629258 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.640088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78574f96-ab6d-4b44-85dc-7ae085b7172b admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.644639 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.645275 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.645551 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.645822 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] PATH_INFO: `/v3/projects/8663a9f0ee474c4dae56510deed7a21d/users/377f51f4d13e4b5ea9b44d7c1a5d1771/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.660578 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8663a9f0ee474c4dae56510deed7a21d, user_id=377f51f4d13e4b5ea9b44d7c1a5d1771, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.663752 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.748691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1449/2898] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:35:21 2022] PUT /identity/v3/projects/8663a9f0ee474c4dae56510deed7a21d/users/377f51f4d13e4b5ea9b44d7c1a5d1771/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:21.752747 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.763829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508956d-7b2d-4150-a6ad-c30be69a6c7c admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.781851 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:21.821011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.821617 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.821896 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.822167 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.822902 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.824499 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b249efd-255b-487f-bcb5-f89b743f65cd admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.834312 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1450/2899] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Aug 9 18:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:35:21.838457 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:21.849586 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3900-c695-43b4-94ef-db3bf54e7a59 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:21.854495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:21.855178 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] REQUEST_METHOD: `PUT` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:21.855482 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:21.855757 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] PATH_INFO: `/v3/projects/8663a9f0ee474c4dae56510deed7a21d/users/377f51f4d13e4b5ea9b44d7c1a5d1771/roles/7409e93d755b445b891c0195b28879f6` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:21.857703 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8663a9f0ee474c4dae56510deed7a21d, user_id=377f51f4d13e4b5ea9b44d7c1a5d1771, role_id=7409e93d755b445b891c0195b28879f6)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:21.860415 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:21.872364 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1450/2900] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Aug 9 18:35:21 2022] PUT /identity/v3/projects/8663a9f0ee474c4dae56510deed7a21d/users/377f51f4d13e4b5ea9b44d7c1a5d1771/roles/7409e93d755b445b891c0195b28879f6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:35:27.765533 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df280-d9e6-480b-bc6f-2f6c7d06daa9 None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:27.766168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df280-d9e6-480b-bc6f-2f6c7d06daa9 None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:27.766168 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18df280-d9e6-480b-bc6f-2f6c7d06daa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:27.797402 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-a18df280-d9e6-480b-bc6f-2f6c7d06daa9 None None] MFA Rules not processed for user `377f51f4d13e4b5ea9b44d7c1a5d1771`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:27.798631 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-a18df280-d9e6-480b-bc6f-2f6c7d06daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:27.869942 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1451/2901] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:27.888468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:27.888468 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:27.971147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621d4b-b089-4633-a90c-a442b50d590f admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:27.977313 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:27.977918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:27.978198 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:27.978471 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:28.050660 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:28.053495 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:28.063702 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1451/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:37.867064 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=53 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:38.059147 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:35:52.730991 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:52.731493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:52.731614 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:52.750688 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] MFA Rules not processed for user `ac1260c6ff5943f4b4e86ea91f232452`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:35:52.751897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:52.758827 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1452/2903] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:35:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:35:52.801841 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:52.803362 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:52.860616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb0c26-0209-44a7-991b-29286314bc10 service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:52.864152 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:52.864746 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:52.865019 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:52.865402 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:52.867169 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:52.880599 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:52.881947 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:52.892731 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1452/2904] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:35:53.868983 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:35:53.926897 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd5db-7830-4119-afc7-8859d0255b0c None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:35:53.930627 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:35:53.931279 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:35:53.931564 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:35:53.931835 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:35:53.933725 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:35:53.948667 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:35:53.950092 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:35:53.961908 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1453/2905] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:35:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:36:02.887795 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=26 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:36:03.956641 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=29 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:36:11.728128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:36:11.738052 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf735e-eddc-4d89-a9bf-6a157ff9d4b0 service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:36:11.741669 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:36:11.742278 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:36:11.742549 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:36:11.742850 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:36:11.751741 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:36:11.753154 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:36:11.762339 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1453/2906] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:36:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:36:21.758385 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:36:49.923458 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:36:49.932380 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775602-42f4-4db5-af6e-11aacaf7a7df service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:36:49.935997 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:36:49.936581 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:36:49.936856 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:36:49.937117 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:36:49.938948 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:36:49.949302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:36:49.950829 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942d909-8f56-4dc6-ac2c-85d31381f51d service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:36:49.959026 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1454/2907] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:36:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:36:59.955330 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:38:30.739784 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:38:30.741374 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:30.761918 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7722-da4d-4797-bfc3-d459c43d9dac service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:38:30.768379 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:38:30.768993 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:38:30.769267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:38:30.769539 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:38:30.771395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:30.785818 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:38:30.787491 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:38:30.807025 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1454/2908] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:38:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:38:32.810559 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:38:32.814434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:38:32.814434 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:38:32.830517 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] MFA Rules not processed for user `ac1260c6ff5943f4b4e86ea91f232452`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:38:32.831767 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:32.838913 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1455/2909] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:38:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:38:33.906222 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:38:33.907848 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:33.978301 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98529cf-3fc8-43a0-a2a1-1f6d32a7d530 service nova] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:38:33.983327 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:38:33.983920 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:38:33.984196 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:38:33.984535 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:38:33.986523 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:33.997743 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:38:33.999338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:38:34.010113 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1455/2910] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:38:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:38:42.825109 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=3 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:38:44.005868 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=38 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:38:53.185054 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:38:53.198353 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec0851-ef89-404e-bd75-1bdd7d36990f None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:38:53.203697 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:38:53.204302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:38:53.204711 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:38:53.205011 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:38:53.206849 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:38:53.218960 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:38:53.221208 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7be142-9efe-4b22-8377-3f3584e372a0 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:38:53.231537 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1456/2911] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:38:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:39:03.226994 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:46.756862 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:46.758279 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:46.769687 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb85e-7e56-406b-ab3d-d3fc139b390d service placement] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:46.775748 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:46.776338 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:46.776616 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:46.776884 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:46.785268 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:46.787461 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:46.796266 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1456/2912] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:56.792291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:58.351442 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] REQUEST_METHOD: `POST` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:58.351999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:58.351999 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:58.371782 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.auth.core [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] MFA Rules not processed for user `ac1260c6ff5943f4b4e86ea91f232452`. Rule list: `[]` (Enabled: `True`). {{(pid=80545) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:58.372955 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:58.379606 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1457/2913] 10.222.0.36 () {58 vars in 958 bytes} [Tue Aug 9 18:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:58.419576 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:58.428521 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e1025-d6fd-4dce-b642-bcfeb7528f8b service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:58.432267 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '886cb33a3f254d20b25936c39b3410d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:58.432853 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:58.433128 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:58.433395 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:58.443992 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:58.445406 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:58.456700 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1457/2914] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:58.746141 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:58.756577 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c947ab5-f257-4ffd-b3a2-c4fca2b6dc7a None None] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:58.760153 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:58.760750 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:58.761022 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:58.761284 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:58.763046 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:58.772611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:58.774034 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:58.782057 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1458/2915] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:59.524206 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:59.533624 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6202bfa-0c37-4964-b8c0-4318c168261e service cinder] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:59.538060 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:59.540306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:59.540306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:59.540306 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:59.549827 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:59.551291 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:59.562574 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1458/2916] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:41:08.778429 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=9 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:09.557428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=12 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:14.470718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:14.484268 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b087ff-d9c2-4a54-968f-80ddf88a63a0 service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:14.489759 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d8460000674bcead45625785d81867', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:14.490373 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:14.490669 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:14.491611 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:14.493613 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:14.505464 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:14.507915 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:14.518313 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1459/2917] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:17.291248 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:17.299932 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4493a6-fa84-4fab-bbeb-3b97fdde4daa service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:17.304214 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:17.304803 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:17.305081 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:17.305350 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:17.314179 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:17.315635 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:17.324983 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1459/2918] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:24.513884 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:27.321036 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=6 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:53.917783 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:53.927164 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0ef73-c4da-4d5a-98d8-87525a69f2f4 service nova] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:53.931294 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:53.932132 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:53.932412 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:53.932683 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:53.940080 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:53.941500 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:53.950129 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1460/2919] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:03.945857 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:32.947414 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:32.957691 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03eda1-a4c2-434d-ac30-63042e023cf2 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:32.960199 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a91c9f4d365a49d7b466292f5078fced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:32.960783 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:32.961062 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:32.961315 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:33.022088 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:33.023560 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:33.033004 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1460/2920] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:36.300292 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:36.302302 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:36.322557 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0aa7fe-a81a-4e84-9940-331bdbcb3c6d service neutron] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:36.327850 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:36.328481 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:36.328760 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:36.329029 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:36.331361 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:36.346274 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:36.347923 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:36.358482 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1461/2921] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:40.182524 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:40.193663 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf27be-ec1d-4ad3-8329-8d6f398ea1f8 service neutron] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:40.202365 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:40.203023 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:40.203300 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:40.203569 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] PATH_INFO: `/v3/users/ac1260c6ff5943f4b4e86ea91f232452` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:40.204705 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1260c6ff5943f4b4e86ea91f232452)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:40.206619 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:40.355655 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1260c6ff5943f4b4e86ea91f232452'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.367355 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1260c6ff5943f4b4e86ea91f232452'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.368670 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1260c6ff5943f4b4e86ea91f232452'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.376000 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1260c6ff5943f4b4e86ea91f232452'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.376840 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1260c6ff5943f4b4e86ea91f232452'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.449428 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1461/2922] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:43:40 2022] DELETE /identity/v3/users/ac1260c6ff5943f4b4e86ea91f232452 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:43:40.690298 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:40.728795 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03615-4c4d-4974-be62-2191d398b793 service placement] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:40.784304 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:40.784928 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:40.785206 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:40.785479 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] PATH_INFO: `/v3/projects/13bae50787744ee98e555d0db03e7c0e` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:40.786633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bae50787744ee98e555d0db03e7c0e)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:40.788768 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:40.901213 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bae50787744ee98e555d0db03e7c0e', 'request_id': 'req-2679af17-e3e0-405f-b0f0-64212f485a6d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.906558 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bae50787744ee98e555d0db03e7c0e', 'request_id': 'req-2679af17-e3e0-405f-b0f0-64212f485a6d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.912841 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bae50787744ee98e555d0db03e7c0e', 'request_id': 'req-2679af17-e3e0-405f-b0f0-64212f485a6d'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.980973 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] The token cache is being invalidate because project 13bae50787744ee98e555d0db03e7c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:43:40.981257 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.981881 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:40.983493 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1462/2923] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:43:40 2022] DELETE /identity/v3/projects/13bae50787744ee98e555d0db03e7c0e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:43:40.987528 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:40.989153 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:41.035968 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eab52-2be5-4092-b337-0c82d5a021c9 admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:41.065690 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:41.092229 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:41.092845 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:41.093122 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:41.093390 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] PATH_INFO: `/v3/users/377f51f4d13e4b5ea9b44d7c1a5d1771` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:41.094502 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=377f51f4d13e4b5ea9b44d7c1a5d1771)` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:41.096442 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:41.257006 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377f51f4d13e4b5ea9b44d7c1a5d1771'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.269555 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377f51f4d13e4b5ea9b44d7c1a5d1771'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.270239 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377f51f4d13e4b5ea9b44d7c1a5d1771'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.277995 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377f51f4d13e4b5ea9b44d7c1a5d1771'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.278630 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.notifications [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377f51f4d13e4b5ea9b44d7c1a5d1771'} {{(pid=80544) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.334580 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1462/2924] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Aug 9 18:43:40 2022] DELETE /identity/v3/users/377f51f4d13e4b5ea9b44d7c1a5d1771 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:43:41.582932 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:41.584386 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:41.598525 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679af17-e3e0-405f-b0f0-64212f485a6d admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:41.649934 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:41.658633 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c800e45fdce14a59ae5bf0f84f3671cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c024e80f00ae4274ad841a20660b3ad2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:41.659280 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:41.659554 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:41.659878 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] PATH_INFO: `/v3/projects/8663a9f0ee474c4dae56510deed7a21d` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:41.661012 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8663a9f0ee474c4dae56510deed7a21d)` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:41.662936 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:41.746794 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8663a9f0ee474c4dae56510deed7a21d', 'request_id': 'req-6b721ae4-7510-417f-9dc6-c4597d930648'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.752231 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8663a9f0ee474c4dae56510deed7a21d', 'request_id': 'req-6b721ae4-7510-417f-9dc6-c4597d930648'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.758883 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8663a9f0ee474c4dae56510deed7a21d', 'request_id': 'req-6b721ae4-7510-417f-9dc6-c4597d930648'} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.849331 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] The token cache is being invalidate because project 8663a9f0ee474c4dae56510deed7a21d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80545) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:43:41.849539 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.850159 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.notifications [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80545) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:43:41.851676 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1463/2925] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Aug 9 18:43:41 2022] DELETE /identity/v3/projects/8663a9f0ee474c4dae56510deed7a21d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:43:51.330886 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:51.867530 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:54.143061 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Authenticating user token {{(pid=80544) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:54.144464 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:54.250109 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf1-064d-4c8c-bf65-414ad503ed6c admin admin] Validating token access rules against request {{(pid=80544) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:54.308356 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80544) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:54.308955 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] REQUEST_METHOD: `GET` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:54.309242 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] SCRIPT_NAME: `/identity` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:54.309512 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80544) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:54.311273 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.fernet_utils [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80544) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:54.322352 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:54.323981 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433213-d127-4b3f-ad3e-7e4878941c9a service placement] RBAC: Authorization granted {{(pid=80544) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:54.333976 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: [pid: 80544|app: 0|req: 1463/2926] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:44:04.329824 dv-cinder-797970-5-iscsi devstack@keystone.service[80544]: DEBUG dbcounter [-] [80544] Writing DB stats keystone:SELECT=74 {{(pid=80544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:57.269718 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Authenticating user token {{(pid=80545) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:57.272542 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.fernet_utils [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80545) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.286867 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b721ae4-7510-417f-9dc6-c4597d930648 admin admin] Validating token access rules against request {{(pid=80545) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:57.293183 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ef94028a4914a688fd4a4539b1ade66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3dc6280ca54d8bb3b695ea7c93bb39', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80545) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:57.301010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] REQUEST_METHOD: `GET` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:57.301010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] SCRIPT_NAME: `/identity` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:57.301010 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80545) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:57.302912 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:57.304574 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c5758-f5f1-4bf1-8813-293cc1b9b071 service placement] RBAC: Authorization granted {{(pid=80545) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:57.315600 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: [pid: 80545|app: 0|req: 1464/2927] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:07.310975 dv-cinder-797970-5-iscsi devstack@keystone.service[80545]: DEBUG dbcounter [-] [80545] Writing DB stats keystone:SELECT=6 {{(pid=80545) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}